UDEV rule help

T

travm

2[H]4U
Joined
Feb 26, 2016
Messages
2,057
I"ve been having a minor issue with my Chromebook(GalliumOS 3.0), it wont save alsa sound settings. I've chased the issue to the alsa-utils startup/shutdown script, reached out to the developer, and was given this as an explanation.
I think that the card may not be available at boot, because it loads the firmware and it may take some time. Use udev rules to resolve this issue like: https://github.com/alsa-project/alsa-utils/blob/master/alsactl/90-alsa-restore.rules.in (the final rules file is available after make)
Click to expand...
Can someone help translate this? Specifically what does he mean by "the final rules file is available after make"?
Do I have to reinstall alsa-utils from source?
Is it enough to copy this file (removing the .in) to /etc/udev/rules.d/?
 
travm said:
I"ve been having a minor issue with my Chromebook(GalliumOS 3.0), it wont save alsa sound settings. I've chased the issue to the alsa-utils startup/shutdown script, reached out to the developer, and was given this as an explanation.

Can someone help translate this? Specifically what does he mean by "the final rules file is available after make"?
Do I have to reinstall alsa-utils from source?
Is it enough to copy this file (removing the .in) to /etc/udev/rules.d/?
Click to expand...
The .in file is a "make" instructions file, I think. You have to build the project with "make" for it to create a working file. There should be instructions somewhere (either in the project's repository or on the website) on how to use "make" to build the project.
 
Nobu said:
The .in file is a "make" instructions file, I think. You have to build the project with "make" for it to create a working file. There should be instructions somewhere (either in the project's repository or on the website) on how to use "make" to build the project.
Click to expand...
So I've gotta build from source eh.
 
travm said:
So I've gotta build from source eh.
Click to expand...
Or find the compiled udev rules file from someone that has already compiled it.

For instance, you could download the package file from https://archlinux.org/packages/extra/x86_64/alsa-utils/ & extract 90-alsa-restore.rules from it.

Option 3 is to install alsa-utils for GalliumOS (apt-get install alsa-utils -or- apt install alsa-utils).

Option 4:

Create /etc/udev/rules.d/90-alsa-restore.rules (sudo nano /etc/udev/rules.d/90-alsa-restore.rules)
You'll still need alsactl from alsa-utils since it's being called in the udev rules.
Code: 
ACTION=="add", SUBSYSTEM=="sound", KERNEL=="controlC*", KERNELS!="card*", TEST=="@sbindir@", TEST=="@mydatadir@", GOTO="alsa_restore_go"
GOTO="alsa_restore_end"

LABEL="alsa_restore_go"
TEST!="@daemonswitch@", RUN+="@sbindir@/alsactl restore $attr{device/number}"
TEST=="@daemonswitch@", RUN+="@sbindir@/alsactl nrestore $attr{device/number}"

LABEL="alsa_restore_end"
 
Last edited:
  • Like
Reactions: Nobu
like this
CrimsonKnight13 said:
Or find the compiled udev rules file from someone that has already compiled it.

For instance, you could download the package file from https://archlinux.org/packages/extra/x86_64/alsa-utils/ & extract 90-alsa-restore.rules from it.

Option 3 is to install alsa-utils for GalliumOS (apt-get install alsa-utils -or- apt install alsa-utils).

Option 4:

Create /etc/udev/rules.d/90-alsa-restore.rules (sudo nano /etc/udev/rules.d/90-alsa-restore.rules)
You'll still need alsactl from alsa-utils since it's being called in the udev rules.
Code: 
ACTION=="add", SUBSYSTEM=="sound", KERNEL=="controlC*", KERNELS!="card*", TEST=="@sbindir@", TEST=="@mydatadir@", GOTO="alsa_restore_go"
GOTO="alsa_restore_end"

LABEL="alsa_restore_go"
TEST!="@daemonswitch@", RUN+="@sbindir@/alsactl restore $attr{device/number}"
TEST=="@daemonswitch@", RUN+="@sbindir@/alsactl nrestore $attr{device/number}"

LABEL="alsa_restore_end"
Click to expand...
Alsa-utils is already installed. I went looking for that rules fine on my system and didn't find it.
 
travm said:
Alsa-utils is already installed. I went looking for that rules fine on my system and didn't find it.
Click to expand...
Aim for extracting it from a pre-compiled package & see how it goes.

Also check in /usr/lib/udev/rules.d
 
travm said:
Just putting the file in etc/udev/rules will have it execute?
Click to expand...
Yes

Code: 
Loading new rules
udev automatically detects changes to rules files, so changes take effect immediately without requiring udev to be restarted. However, the rules are not re-triggered automatically on already existing devices. Hot-pluggable devices, such as USB devices, will probably have to be reconnected for the new rules to take effect, or at least unloading and reloading the ohci-hcd and ehci-hcd kernel modules and thereby reloading all USB drivers.

If rules fail to reload automatically:
# udevadm control --reload

To manually force udev to trigger your rules:
# udevadm trigger

Source: https://wiki.archlinux.org/title/Udev
 
  • Like
Reactions: travm
like this
So I did find the rule, its in /lib/udev/rules.d
Why does this not work then?
 
Do you have the alsarestore service enabled? (Not sure if correct name, but something like that)

Edit: asking because that rule seems to rely on it starting and stopping
 
  • Like
Reactions: travm
like this
That looks to be at least part of the issue.

Yay more troubleshooting.

If anything of this is obvious to you what I need to do to fix, i'm all ears.
Code: 
travis@galliumos:~$ sudo service alsa-restore status
● alsa-restore.service - Save/Restore Sound Card State
   Loaded: loaded (/lib/systemd/system/alsa-restore.service; static; vendor pres
   Active: inactive (dead)
Condition: start condition failed at Mon 2021-10-04 18:44:17 EDT; 7min ago
           └─ ConditionPathExistsGlob=/dev/snd/control* was not met
     Docs: man:alsactl(1)
lines 1-6/6 (END)

I'm also trying to follow up with the dev, but I'm afraid he's shoved this aside as not his issue. I believe it is though.

/dev/snd/ has two files in it that meet the apparent criteria above, ControlC0 and ControlC1. Could the cap C be the issue in the service code?

I suppose to be fair I should install the current version from source 1.2.5.1 instead of 1.1.3.1 that is packaged with the distro....
 
Last edited:
More;
https://patchwork.kernel.org/project/alsa-devel/patch/[email protected]/
Still no solution. I have tried changing "After=system.init" to "After=sound.target" in alsa-state.service, and in alsa-restore.service. Neither seems to get it to start. Looks like the issue has been present since 2016.
not sure what i'm doing as far as systemd is concerned.

So commenting out the items removed from both files in that patch submission allows the service to start on boot.
It still does't save settings.

This looks a little bigger than I thought. I guess i'll kick the GalliumOS people and see if they can sort out whos bug this is.

alsa-restore.service
Code: 
#
# Note that two different ALSA card state management schemes exist and they
# can be switched using a file exist check - /etc/alsa/state-daemon.conf .
#

[Unit]
Description=Save/Restore Sound Card State
Documentation=man:alsactl(1)
ConditionPathExists=!/etc/alsa/state-daemon.conf
#ConditionPathExistsGlob=/dev/snd/control*
#After=alsa-state.service

[Service]
Type=oneshot
RemainAfterExit=true
ExecStartPre=/bin/mkdir -p /run/alsa
ExecStart=-/usr/sbin/alsactl -E HOME=/run/alsa restore
ExecStop=-/usr/sbin/alsactl -E HOME=/run/alsa store

alsa-state.service
Code: 
#
# Note that two different ALSA card state management schemes exist and they
# can be switched using a file exist check - /etc/alsa/state-daemon.conf .
#

[Unit]
Description=Manage Sound Card State (restore and store)
Documentation=man:alsactl(1)
ConditionPathExists=/etc/alsa/state-daemon.conf
#After=sysinit.target

[Service]
Type=simple
ExecStartPre=/bin/mkdir -p /run/alsa
ExecStart=-/usr/sbin/alsactl -E HOME=/run/alsa -s -n 19 -c rdaemon
ExecStop=-/usr/sbin/alsactl -E HOME=/run/alsa -s kill save_and_quit
 
Last edited:
  • Like
Reactions: Nobu
like this
Looks like you're on the right track. My Linux box is currently living at my brother's, so I can't help much more on this right now. If you still need help this weekend, ping me. I have Arch installed, so I should at least be able to see if I can repro and maybe install the git version.

If you do get the latest, be sure to grab an updated kernel as well (keep the old as backup, though). Alsa lives in the kernel, so having newer user space may cause problems.
 
Looks like GalliumOS isn't finding whatever driver or device it requires to work.
Code: 
ConditionPathExistsGlob=/dev/snd/control* was not met

Does your Chromebook work with boot modification like SeaBIOS for booting any x64 OS? https://mrchromebox.tech/
Personally, I would look into newer Linux distros & modify one to your liking.
 
CrimsonKnight13 said:
Looks like GalliumOS isn't finding whatever driver or device it requires to work.
Code: 
ConditionPathExistsGlob=/dev/snd/control* was not met

Does your Chromebook work with boot modification like SeaBIOS for booting any x64 OS? https://mrchromebox.tech/
Personally, I would look into newer Linux distros & modify one to your liking.
Click to expand...
I have done that, the issue is the damn keyboard. I tried mint specifically, and while it worked, I couldnt install the correct keymap for the chromebook keyboard. It also had the exact same sound issue. As did vanilla ubuntu. iirc vanilla ubuntu "may" have had a working chromebook keyboard layout. But I dont like vanilla ubuntu.

Yes, i'm using SeaBIOS.

I am tempted to give arch a shot, but i've spent many years learning bits of ubuntu/debian specific things. Would be a big jump. I hate having to wade through man pages, scripts, and conf files and much prefer it when my stuff just works and I can use it for its intended purpose.
 
travm said:
I have done that, the issue is the damn keyboard. I tried mint specifically, and while it worked, I couldnt install the correct keymap for the chromebook keyboard. It also had the exact same sound issue. As did vanilla ubuntu. iirc vanilla ubuntu "may" have had a working chromebook keyboard layout. But I dont like vanilla ubuntu.

Yes, i'm using SeaBIOS.

I am tempted to give arch a shot, but i've spent many years learning bits of ubuntu/debian specific things. Would be a big jump. I hate having to wade through man pages, scripts, and conf files and much prefer it when my stuff just works and I can use it for its intended purpose.
Click to expand...
If you do decide to go Arch, you could try Manjaro, EndeavourOS, or SalientOS.

For GalliumOS, you're on 3, right? You could always work with the newest version (4?) to see if anything changes.

Here's more info on Chromebook support https://wiki.archlinux.org/title/Chrome_OS_devices

Also, as Nobu suggested, might be a kernel issue. Some people maintain kernels with more driver modules built-in for Chrome devices. You could always use DKMS modules to add more support as well.
 
Yeah, its annoying as hell. I just upgraded my USB disk to a 128gb model, and switched from Mint (mainly due to the keyboard not working properly, along with the sound issue). I should have jumped into arch off the bat. It takes about a day and a half to download and install the software I use on this thing.
 
travm said:
Yeah, its annoying as hell. I just upgraded my USB disk to a 128gb model, and switched from Mint (mainly due to the keyboard not working properly, along with the sound issue). I should have jumped into arch off the bat. It takes about a day and a half to download and install the software I use on this thing.
Click to expand...
What sort of a funky keyboard do you use? I've never seen one so far that wouldn't work on linux lol.
 
B00nie said:
What sort of a funky keyboard do you use? I've never seen one so far that wouldn't work on linux lol.
Click to expand...
Chromebook keyboards are weird. The top keys function as function keys & there are ways to map them for their alternate functions.
acer-c720-chromebook-keyboard-key-replacement.png
 
Last edited:
CrimsonKnight13 said:
Chromebook keyboards are weird. The top keys function as function keys & there are ways to map them for their alternate functions.
View attachment 400714
Click to expand...
its not just that, but delete is super-alt + backspace. The software i use really needs a delete key, and being able to alt+backspace suits me just fine. Except not all distro's include a layout that works. Mint had no functioning layouts at all, just different languages.
Manjaro XFCE seems to workish, but alsamixer doesnt work (so same problem exists), and the brightness keys dont work. I'll see if the people on the manjaro forum can be more helpful. I might try manjaro KDE before i slide it in the trash too and go back to galliumos.
 
travm said:
its not just that, but delete is super-alt + backspace. The software i use really needs a delete key, and being able to alt+backspace suits me just fine. Except not all distro's include a layout that works. Mint had no functioning layouts at all, just different languages.
Manjaro XFCE seems to workish, but alsamixer doesnt work (so same problem exists), and the brightness keys dont work. I'll see if the people on the manjaro forum can be more helpful. I might try manjaro KDE before i slide it in the trash too and go back to galliumos.
Click to expand...
Use https://wiki.archlinux.org/title/Chrome_OS_devices & https://wiki.archlinux.org/title/Laptop to possibly resolve some of your issues.
 
  • Like
Reactions: travm
like this
CrimsonKnight13 said:
Use https://wiki.archlinux.org/title/Chrome_OS_devices & https://wiki.archlinux.org/title/Laptop to possibly resolve some of your issues.
Click to expand...
I have been through them. It's a pity, I can only ever get to about 75%, and updates continually fix one thing, break two others. I will go a little deeper tonight or tomorrow. Then possibly on to KDE, or perhaps just vanilla Arch and I'll install my own shit. Did it once with Gentoo years back and it worked great. Tried with debian later and it was not so great
 
travm said:
I have been through them. It's a pity, I can only ever get to about 75%, and updates continually fix one thing, break two others. I will go a little deeper tonight or tomorrow. Then possibly on to KDE, or perhaps just vanilla Arch and I'll install my own shit. Did it once with Gentoo years back and it worked great. Tried with debian later and it was not so great
Click to expand...
Manjaro can be wacky with delayed updates. You might want to try one of the others I mentioned.

Also, what Chromebook model do you have?
 
CrimsonKnight13 said:
Manjaro can be wacky with delayed updates. You might want to try one of the others I mentioned.

Also, what Chromebook model do you have?
Click to expand...
acer cb3-131.
What is most frusterating is that each distribution I try, might fix the issue i'm trying to have fixed, but then something else, or two other things are broken. Its like an excercise in choosing the least broken OS.
 
travm said:
acer cb3-131.
What is most frusterating is that each distribution I try, might fix the issue i'm trying to have fixed, but then something else, or two other things are broken. Its like an excercise in choosing the least broken OS.
Click to expand...
I've had varyingly strange issues with Linux over the years just like Windows (especially with drivers), so I know how that feels. I do my best as well, so I hope you'll find a good place to be with some OS.
 
CrimsonKnight13 said:
I've had varyingly strange issues with Linux over the years just like Windows (especially with drivers), so I know how that feels. I do my best as well, so I hope you'll find a good place to be with some OS.
Click to expand...
Thanks for the help and kind words.

I understand I'm trying to work with fringe hardware on this; I have a box in workshop that tends to "just work" in whatever OS I throw at it. It has more mundane hardware. The chromebook was for 3 things;
1. cheap AF
2. small, light, easy on the lap on the couch.
3. Can run some dev software and CAD, specifically MPlabX (which it does, fabulously), and KiCAD (needs the super-alt + backspace, delete key combo to work or I will wear out the trackpad)

It does all of these things, and well. Its just minor annoyances when i try to do some things outside of the main reasons I bought it. Sound issues affect video calls, brightness keys not working is annoying, only because I do tend to twiddle that often, kept as low as possible. I actually really like the brightness keys and might almost want them on the desktop for that reason. Relax eyes!

As stated, I'm going to load up on some ISO's and spend a half day blasting through them to see who's defaults come closest. Likely even start with vanilla arch and build the system from scratch.
 
travm said:
Thanks for the help and kind words.

I understand I'm trying to work with fringe hardware on this; I have a box in workshop that tends to "just work" in whatever OS I throw at it. It has more mundane hardware. The chromebook was for 3 things;
1. cheap AF
2. small, light, easy on the lap on the couch.
3. Can run some dev software and CAD, specifically MPlabX (which it does, fabulously), and KiCAD (needs the super-alt + backspace, delete key combo to work or I will wear out the trackpad)

It does all of these things, and well. Its just minor annoyances when i try to do some things outside of the main reasons I bought it. Sound issues affect video calls, brightness keys not working is annoying, only because I do tend to twiddle that often, kept as low as possible. I actually really like the brightness keys and might almost want them on the desktop for that reason. Relax eyes!

As stated, I'm going to load up on some ISO's and spend a half day blasting through them to see who's defaults come closest. Likely even start with vanilla arch and build the system from scratch.
Click to expand...
Find this page? https://wiki.archlinux.org/title/Acer_CB3-131_Chromebook

Hopefully that one has something in it that's useful.
 
  • Like
Reactions: travm
like this
CrimsonKnight13 said:
Find this page? https://wiki.archlinux.org/title/Acer_CB3-131_Chromebook

Hopefully that one has something in it that's useful.
Click to expand...
I might have to add to that page when I get this sorted. Its painfully out of date. Those issues went away with Kernel updates, and are over two years old. But like I said, fix one thing, break two others, has been the progress. Still my core reasons for having this thing still work. Just trying to make it better.
Thanks again
 
travm said:
I might have to add to that page when I get this sorted. Its painfully out of date. Those issues went away with Kernel updates, and are over two years old. But like I said, fix one thing, break two others, has been the progress. Still my core reasons for having this thing still work. Just trying to make it better.
Thanks again
Click to expand...
You might be an expert with your Chromebook & Linux after you've walked that hellscape. Clonezilla your perfect build when that happens. :LOL:
 
travm said:
I"ve been having a minor issue with my Chromebook(GalliumOS 3.0), it wont save alsa sound settings. I've chased the issue to the alsa-utils startup/shutdown script, reached out to the developer, and was given this as an explanation.
Click to expand...

How did you determine the cause?

Do you have systemd installed? PulseAudio?

[Edit: I missed some of your later posts, which answered the systemd question. Did you decide to switch distros? Uncertain.]

travm said:
Can someone help translate this?
Click to expand...

The rules file needs to be tailored to your distribution, so the result probably won't be what you want. That said, this is what I get from substituting variables found in the project files:

Code: 
ACTION=="add", SUBSYSTEM=="sound", KERNEL=="controlC*", KERNELS!="card*", TEST=="/usr/sbin", TEST=="/usr/share/alsa", GOTO="alsa_restore_go"
GOTO="alsa_restore_end"

LABEL="alsa_restore_go"
TEST!="/etc/alsa/state-daemon.conf", RUN+="/usr/sbin/alsactl restore $attr{device/number}"
TEST=="/etc/alsa/state-daemon.conf", RUN+="/usr/sbin/alsactl nrestore $attr{device/number}"

LABEL="alsa_restore_end"

However, your distribution's package will probably look slightly different. For example, this is from the current stable release of Debian (Bullseye):

Code: 
# /usr/lib/udev/rules.d/90-alsa-restore.rules

ACTION=="add", SUBSYSTEM=="sound", KERNEL=="controlC*", KERNELS!="card*", TEST=="/usr/sbin", TEST=="/usr/share/alsa", GOTO="alsa_restore_go"
GOTO="alsa_restore_end"

LABEL="alsa_restore_go"
TEST!="/etc/alsa/state-daemon.conf", TEST=="/usr/sbin/alsactl", RUN+="/usr/sbin/alsactl -E HOME=/run/alsa -E XDG_RUNTIME_DIR=/run/alsa/runtime restore $attr{device/number}"
TEST=="/etc/alsa/state-daemon.conf", TEST=="/usr/sbin/alsactl", RUN+="/usr/sbin/alsactl -E HOME=/run/alsa -E XDG_RUNTIME_DIR=/run/alsa/runtime nrestore $attr{device/number}"

LABEL="alsa_restore_end"

travm said:
Specifically what does he mean by "the final rules file is available after make"?
Click to expand...

The joy of GNU Autotools!
 
Last edited:
ElementDave said:
How did you determine the cause?

Do you have systemd installed? PulseAudio?

[Edit: I missed some of your later posts, which answered the systemd question. Did you decide to switch distros? Uncertain.]



The rules file needs to be tailored to your distribution, so the result probably won't be what you want. That said, this is what I get from substituting variables found in the project files:

Code: 
ACTION=="add", SUBSYSTEM=="sound", KERNEL=="controlC*", KERNELS!="card*", TEST=="/usr/sbin", TEST=="/usr/share/alsa", GOTO="alsa_restore_go"
GOTO="alsa_restore_end"

LABEL="alsa_restore_go"
TEST!="/etc/alsa/state-daemon.conf", RUN+="/usr/sbin/alsactl restore $attr{device/number}"
TEST=="/etc/alsa/state-daemon.conf", RUN+="/usr/sbin/alsactl nrestore $attr{device/number}"

LABEL="alsa_restore_end"

However, your distribution's package will probably look slightly different. For example, this is from the current stable release of Debian (Bullseye):

Code: 
# /usr/lib/udev/rules.d/90-alsa-restore.rules

ACTION=="add", SUBSYSTEM=="sound", KERNEL=="controlC*", KERNELS!="card*", TEST=="/usr/sbin", TEST=="/usr/share/alsa", GOTO="alsa_restore_go"
GOTO="alsa_restore_end"

LABEL="alsa_restore_go"
TEST!="/etc/alsa/state-daemon.conf", TEST=="/usr/sbin/alsactl", RUN+="/usr/sbin/alsactl -E HOME=/run/alsa -E XDG_RUNTIME_DIR=/run/alsa/runtime restore $attr{device/number}"
TEST=="/etc/alsa/state-daemon.conf", TEST=="/usr/sbin/alsactl", RUN+="/usr/sbin/alsactl -E HOME=/run/alsa -E XDG_RUNTIME_DIR=/run/alsa/runtime nrestore $attr{device/number}"

LABEL="alsa_restore_end"



The joy of GNU Autotools!
Click to expand...
The udev rules for alsa simply don't work. There's clearly a bug from 2016 that prevents the necessary service from running, but even fixing that still does not cause the udev rules to work. Distro shopping right now.
 
travm said:
The udev rules for alsa simply don't work. There's clearly a bug from 2016 that prevents the necessary service from running, but even fixing that still does not cause the udev rules to work. Distro shopping right now.
Click to expand...
I wish I'd seen your thread earlier. Let me know if you haven't completely given up on fixing the problem, but good luck whatever you decide.

If your audio is working minus the save/restore issue at boot, you're probably a trivial fix away from a solution.

Does HOME=/run/alsa /usr/sbin/alsactl restore work after your system is up and running?

Based on your later posts, it seems the device nodes for your sound aren't created before the restore service is executed.

travm said:
So commenting out the items removed from both files in that patch submission allows the service to start on boot.
It still does't save settings.

alsa-restore.service
Code: 
#
# Note that two different ALSA card state management schemes exist and they
# can be switched using a file exist check - /etc/alsa/state-daemon.conf .
#

[Unit]
Description=Save/Restore Sound Card State
Documentation=man:alsactl(1)
ConditionPathExists=!/etc/alsa/state-daemon.conf
#ConditionPathExistsGlob=/dev/snd/control*
#After=alsa-state.service
Click to expand...
Commenting out those lines ensures that the service will start provided that /etc/alsa/state-daemon.conf does not exist, but the device node(s) in ConditionPathExistsGlob need to be present prior to starting the service otherwise it won't have the intended effect.
travm said:
alsa-state.service
Click to expand...

You can safely ignore the alsa-state service. The alsa-restore and alsa-state service are mutually exclusive.

What do your actual udev rules look like (from /lib/udev/rules.d/90-alsa-restore.rules)? The rules posted early in the thread won't work.
 
ElementDave
The cause of the problem seems to be that the sound hardware isn't available until the firmware is loaded. This delay seems to throw off the auto restore. This should be solved by udev rules, but I don't understand alsa or udev rules in general well enough to troubleshoot.
I have already removed gallium, and manjaro xfce has more issues. Going to try a couple other distros, I would like to run cinnamon I think.
I'll tag you in when I get settled on a distro, every single one I've tried so far has minor sound issues.
 
travm said:
ElementDave
The cause of the problem seems to be that the sound hardware isn't available until the firmware is loaded. This delay seems to throw off the auto restore. This should be solved by udev rules, but I don't understand alsa or udev rules in general well enough to troubleshoot.
I have already removed gallium, and manjaro xfce has more issues. Going to try a couple other distros, I would like to run cinnamon I think.
I'll tag you in when I get settled on a distro, every single one I've tried so far has minor sound issues.
Click to expand...
You might be able to speed up the process by including the firmware in the initrd, I'm not sure. For Arch, see:
https://wiki.archlinux.org/title/mkinitcpio
https://man.archlinux.org/man/mkinitcpio.conf.5

Specifically, I think if you include the fw file in the "files" field, it should be able to load the firmware earlier in the boot process.

Edit: this should happen automatically on Arch, but it may not on other distros, or if it doesn't detect it properly, or however it figures that out.
 
You must log in or register to reply here.
Back
Top