Monday, November 28, 2016

Switching from IDE to AHCI on an ICH7-M based laptop

Many older laptops have SATA controllers and SATA drives, but set up the chipset in IDE mode instead of AHCI mode. Compared to IDE mode, AHCI can provide faster transfer rates, NCQ and power savings. It may not be possible to select AHCI mode in the BIOS, but it can be done by writing to PCI configuration registers directly. The actual IDE (PATA) ports are not supported in AHCI mode, so a PATA DVD drive would not be usable when AHCI is enabled.

You can read and write PCI registers using setpci in Linux, but if you want to make this change, it must be done before the device is detected. One way to do it is via a kernel patch, but recompiling the kernel every time for updates is kind of annoying. Another way is via GRUB's setpci command. This is very easy, and works well, but breaks resume from sleep, because GRUB doesn't run then and the kernel will encounter the device in IDE mode.

How to do this via GRUB

First, you need to find your controller. Use lspci and look at the output. On my Inspiron 6400 I find "00:1f.2 IDE interface: Intel Corporation 82801GBM/GHM (ICH7-M Family) SATA Controller [IDE mode] (rev 01)". The 00:1f.2 is the device's identifier in Linux, and lspci -n | grep 00:1f.2 shows "00:1f.2 0101: 8086:27c4 (rev 01)" This gives you the PCI ID of the device: 8086:27c4.

According to the ICH-7 datasheet PDF, you need to set the 8-bit MAP—Address Map Register at offset 0x90 to 0x40. (A patch I had before also set the SCRAE bit in the register at offset 0x94, but that just provides access to some AHCI registers when AHCI is not enabled, and is irrelevant when AHCI is enabled.) This means the following GRUB command is needed.

setpci -d 8086:27c4 90.b=40

You can press e when the GRUB menu is displayed, and add it to the end of the command that boots Linux. Adding it earlier caused a hang, probably because the BIOS cannot handle AHCI mode, and so it needs to be set after the kernel and initrd are loaded. After finding this works, I added an /etc/grub.d/42_ahci file with a stripped down Linux menu entry taken from /boot/grub/grub.cfg. It would be better if I could call the Ubuntu menu entry as a subroutine, but that doesn't seem possible with GRUB.

#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry 'Ubuntu AHCI' --class ubuntu --class gnu-linux --class gnu --class os {
recordfail
insmod ext2
set root='hd0,msdosPARTITION_NUMBER_HERE'
linux /vmlinuz root=UUID=PUT_UUID_HERE ro resume=SWAP_PARTITION
initrd /initrd.img
setpci -d 8086:27c4 90.b=40
}

The Results

This change increases sudo nice -n -19 dd if=/dev/sda bs=1M of=/dev/null count=1000 speed from 128 MB/s to 141 MB/s with a PNY SSD2SC120G1CS1754D117-551 SSD. It also enables NCQ, and causes the DVD drive to not be detected at all. Here are the IDE lines which disappear from dmesg. Note that this is based on diff output and is just a list of lines in order, not a continuous section.

pci 0000:00:1f.2: [8086:27c4] type 00 class 0x010180
pci 0000:00:1f.2: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
pci 0000:00:1f.2: legacy IDE quirk: reg 0x14: [io 0x03f6]
pci 0000:00:1f.2: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
pci 0000:00:1f.2: legacy IDE quirk: reg 0x1c: [io 0x0376]
ata_piix 0000:00:1f.2: version 2.13
ata_piix 0000:00:1f.2: MAP [ P0 P2 IDE IDE ]
scsi host0: ata_piix
scsi host1: ata_piix
ata1: SATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0xbfa0 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xbfa8 irq 15
ata2.00: ATAPI: SONY DVD+/-RW DW-Q58A, UDS2, max UDMA/33
ata1.00: 234441648 sectors, multi 2: LBA48 NCQ (depth 0/32)
ata2.00: configured for UDMA/33
scsi 1:0:0:0: CD-ROM SONY DVD+-RW DW-Q58A UDS2 PQ: 0 ANSI: 5
sr 1:0:0:0: [sr0] scsi3-mmc drive: 24x/24x writer cd/rw xa/form2 cdda tray
cdrom: Uniform CD-ROM driver Revision: 3.20
sr 1:0:0:0: Attached scsi CD-ROM sr0
sr 1:0:0:0: Attached scsi generic sg1 type 5


Here are the new lines which appear instead:

pci 0000:00:1f.2: [8086:27c5] type 00 class 0x010601
pci 0000:00:1f.2: reg 0x24: [mem 0x00000000-0x000003ff]
pci 0000:00:1f.2: BAR 5: assigned [mem 0x80000000-0x800003ff]
ahci 0000:00:1f.2: version 3.0
ahci 0000:00:1f.2: enabling device (0005 -> 0007)
ahci 0000:00:1f.2: forcing PORTS_IMPL to 0xf
ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
ahci 0000:00:1f.2: AHCI 0001.0100 32 slots 4 ports 1.5 Gbps 0xf impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq ilck stag pm led clo pmp pio slum part
scsi host0: ahci
scsi host1: ahci
scsi host2: ahci
scsi host3: ahci
ata1: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000100 irq 27
ata2: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000180 irq 27
ata3: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000200 irq 27
ata4: SATA max UDMA/133 abar m1024@0x80000000 port 0x80000280 irq 27
ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata1.00: 234441648 sectors, multi 2: LBA48 NCQ (depth 31/32), AA
ata2: failed to resume link (SControl 0)
ata2: SATA link down (SStatus 0 SControl 0)
ata3: SATA link down (SStatus 0 SControl 300)
ata4: failed to resume link (SControl 0)
ata4: SATA link down (SStatus 0 SControl 0)

A better solution

Another alternative, which can support suspend to RAM, is to set registers from an altered DSDT. Read about the DSDT changes here and read about how to use a custom DSDT in Linux here. You first need to dump your DSDT, decompile it and fix it so it can compile properly. People do this sort of thing for running Mac OS on a PC, and you can find plenty of info on sites relating to that. Then you can add your modifications. In Ubuntu, simply copy the compiled dsdt.aml to /boot and run update-grub.

Tuesday, November 22, 2016

Sharing an IR receiver via an opto-isolator

I wanted to share an infrared remote receiver between the Pace DC550D cable box and Raspberry Pi 2 B running Kodi. This way there is no need to mount another visible IR receiver. It turned out to be a harder than anticipated because I used the wrong opto-isolator.

IR Receivers

An IR receiver has 3 connections: ground, power and signal output. With many IR receivers, the output is open collector and active low. This means when there is no remote signal detected, the output is not driven, and when a signal is detected, it is driven low. In order to actually see a voltage change at the output, a pull-up resistor is needed. Then the output stays high with no signal and goes low when a signal is detected.

The Pace DC550D, like many devices, connects to the IR receiver via a standard headphone jack. The tip supplies 5 V power, the ring (middle contact) is the signal output, and the sleeve is ground. This arrangement makes the most sense, but I don't know if every other device uses it.

The signal stays at just over 3 V normally and goes low when IR is detected. This might be because it connects to 3.3 V device. With an open collector output, the pull-up resistor does not need to connect to the same voltage as the device's power supply. So, an IR receiver powered by 5 V and with the pull-up resistor connected to 3.3 V will properly interface with 3.3 V devices.

Opto-isolation

Accessing the IR receiver signal is easy via headphone splitter and headphone plug. It ought to be possible to connect the IR receiver directly to the Raspberry Pi. Everything is grounded together via HDMI, though also connecting grounds would be good. However, I wanted opto-isolation for protection.

I connected the opto-isolator LED between 5 V power and signal output. This meant the LED would come on when a signal is detected. This seems better because it is drawing power through the IR receiver output, not the pull-up resistor. Though there is a problem because the off state is just over 3 V and LED voltage drop is only 1 V. I added 3 diodes in series to deal with this. A current limiting resistor is also required for the LED.

At first I tried to just connect the opto-isolator output to ground and a GPIO pin, and use the Raspberry Pi's internal pull-up. First, this did not work at all, because the lirc_rpi module parameter for setting that is ignored. Pull up can either be enabled by running a short Python program every time, or by by adding a dtparam=gpio_in_pull=up line after the dtoverlay=lirc-rpi line in /boot/config.txt. With the pull-up enabled, I got a signal, but it was useless.

The problem was the turn-off time of the optocoupler. I chose a TIL113 with photodarlington output, so that I would need to draw less power from the IR receiver. However, photodarlington opto-isloators have very slow shutoff. Adding a base resistor didn't help much, and even with a pull-up to 3.3 V on the output, LIRC was unreliable. Eventually I changed to a PS2010 with a just a normal phototransistor. Due to its much lower current transfer ratio, I had to carefully choose LED current limiting and output pull-up resistors. The output pull-up was still necessary; the Raspberry Pi's internal pull-up did not give a good enough signal.

The Results

This seems fully reliable now. I'm using a Harmony 300 remote, with the device set to Microsoft Media Player [Microsoft Xbox Media Center Extender]. It maps to the mceusb LIRC remote. The MyHarmony software doesn't configure all the possible keys, so it is necessary to edit the key configuration. The Prev button, corresponding to KEY_BACK is especially important for the Kodi interface.

Monday, November 21, 2016

Connecting to WiFi using Debian's /etc/network/interfaces

It's fairly easy to connect to a wireless network from the command line using /etc/network/interfaces if you know what you're doing. When connecting to a network with WPA or WPA2 encryption, you will need wpa_supplicant, but you don't need to create a conf file. Instead, options corresponding to wpa_supplicant.conf options can be placed directly in /etc/network/interfaces. Those options aren't documented in the man page, but you can find them in /usr/share/doc/wpasupplicant/examples/wpa_supplicant.conf.gz . Add a wpa- prefix and change the underscore to a dash. For example, scan_ssid in wpa_supplicant.conf becomes wpa-scan-ssid in /etc/network/interfaces. Something like this was suggested elsewhere.

iface wlan0 inet dhcp
    wpa-scan-ssid 1
    wpa-ap-scan 1
    wpa-key-mgmt WPA-PSK
    wpa-proto RSN WPA
    wpa-pairwise CCMP TKIP
    wpa-group CCMP TKIP
    wpa-ssid "<your ssid>"
    wpa-psk "<your password>

You only need wpa-scan-ssid 1 if you want to connect to a non-broadcasting network. Some of the other parameters have sensible defaults. Note that RSN is WPA2 and CCMP is the most secure encryption. The wpa-ssid quotes will be stripped, so you can use a SSID with leading and/or trailing spaces. This is not possible with wireless-essid.

You can also use other /etc/network/interfaces options, like an allow-hotplug wlan0 line so the interface gets configured when you plug in the dongle, or a static IP instead of DHCP. The interfaces(5) man page only lists a few of the options. Others are provided by shell scripts installed by other packages, like /etc/wpa_supplicant/functions.sh for the wpa- options.

Once everything is done, you can use sudo ifup wlan0 to connect. For some reason, that command can become unresponsive to attempts to kill it, and the ifup process may need to be killed elsewhere. Use sudo ifdown wlan0 to disconnect.

Thursday, November 17, 2016

Using Pidgin and purple-facebook for Facebook chat may also cause problems. App passwords may help.

Earlier I had problems with the Miranda NG Facebook plugin causing captchas on Facebook. I would get captchas even when posting and sharing very obviously innocent stuff. The plugin would get updated, and captchas would stop, but then they would start again later. So, I gave up on that and started using Pidgin with purple-facebook in Windows as well.

I had been using Pidgin with Purple Facebook in Linux for a while, but when I started using it in Windows, Facebook reacted as if my account had been hacked. I was forced to change my password, and examine recent activity. This problem repeated. I don't know if it's related to use of purple-facebook in Windows or use of it in too many places or a newer version or what. I reported the bug, and other users responded that they also encountered the problem.

Facebook has an app passwords feature which allows you to auto-generate passwords and use those passwords for apps instead of your main Facebook password. I'm now using that feature, with separate app passwords for every different Pidgin installation. So far so good.

Oh, and by the way, if Facebook forces you to change your password, don't bother trying to re-use the old password. They seem to blacklist the passwords used when their system thinks your account was hacked.

How to obtain higher resolution photos by altering the image URL

Often websites scale down the photos which they display. If you carefully examine the image URL and change it, you may be able to increase the resolution and quality. Look for parameters which specify the size and try to change them or strip them away. Also, if the photo URL refers to another path, try to directly access that other path.

You can often get an image URL by right clicking on the image. If that does not work, use your browser's developer tools or the media tab of page info in Firefox. The inspect feature of developer tools should make finding the image fairly easy.

Here are a few examples of these transformations:

http://wpmedia.windsorstar.com/2016/11/1d1.jpg?quality=55&strip=all&w=840&h=630&crop=1
http://wpmedia.windsorstar.com/2016/11/1d1.jpg

This was pretty obvious. Remove the parameters and get the original image size. It also seems to avoid JPEG re-encoding, and the accompanying increase in artifacts.

https://i.cbc.ca/1.3929515.1484163948!/fileImage/httpImage/image.jpg_gen/derivatives/16x9_460/pillar-lights.jpg
https://i.cbc.ca/1.3929515.1484163948!/fileImage/httpImage/image.jpg

The parameters do not necessarily start with a question mark.

http://www.amherstburg.ca/ThumbGen.ashx?%2fAreas%2fCustom%2fContentFiles%2fportrait+of+ferry+and+shoreline.png/600/452/w
http://www.amherstburg.ca/Areas/Custom/ContentFiles/portrait%20of%20ferry%20and%20shoreline.png

In this case, you see another path given as a parameter to ThumbGen.ashx. You need to remove the encoding of that path. Note that the path is encoded with percent encoding and + instead of spaces.

http://i2.cdn.cnn.com/cnnnext/dam/assets/161117110259-dutch-war-ship-exlarge-169.jpg
http://i2.cdn.cnn.com/cnnnext/dam/assets/161117110259-dutch-war-ship.jpg

In this case part of the "file name" was the culprit, and the change removes cropping and text.

https://images-na.ssl-images-amazon.com/images/I/51dIUaKqkYL._SY500_.jpg
https://images-na.ssl-images-amazon.com/images/I/51dIUaKqkYL.jpg

Again, the last part of the file name before the .jpg had to be removed.

https://c1.staticflickr.com/6/5722/22981183464_8c7035afbb_b.jpg
https://c1.staticflickr.com/6/5722/22981183464_8c7035afbb_h.jpg

For some sites, there are site-specific tricks which are not self-explanatory. On Flickr, try changing the last letter after the underscore to h. In the past o was needed for some photos and may still be needed sometimes.

http://s7d5.scene7.com/is/image/CanadianTire/0431755_1?defaultImage=image_na_EN&wid=160&hei=160&op_sharpen=1
http://s7d5.scene7.com/is/image/CanadianTire/0431755_1?defaultImage=image_na_EN&wid=1600&hei=1600&op_sharpen=1

In this case, removing the parameters returns the default small size, but you can change the parameters to increase image size. Note that it would be almost entirely pointless to increase the size beyond the original size. You would just get a larger file without added detail. On some sites, if you specify a size that's larger than the original size, you will get the original size.

If you are not satisfied with what you can obtain at one site, then try Google search by image or TinEye. Remember that even if the search engine doesn't find bigger images, another site may allow you to download a larger image by modifying the URL.