Electrocuting Modern Elephants

Finally my position about SOPA, ACTA, PIPA and friends… it was about time to jump into the flame war, wasn’t it?

Read More

Electrocuting Modern Elephants

Electrocuting Modern Elephants

Even as a member of the IEEE, an Institute whose origins date back to Thomas Alva Edison‘s work as a visionary inventor, I must admit the guy had some very serious issues when dealing with business. He stole, cheated, bribed but one of the most impressive things I learned about Edison was that he electrocuted animals in public. He would do so to exemplify that Alternate Current, a new development at that time that promised cheaper electricity for wider areas, was extremely dangerous and could kill humans (he later supported the idea of the electric chair for the same purpose), and he would do that because he had almost full control over Direct Current distribution in the United States at that time, so he really wanted to keep people away from AC. This was called the War of Currents and a notable killing of that time was that of Topsy the Elephant, which was even caught on tape and remains as a shocking proof of this period of industrial evolution.

Topsy the Elephant after the electrocution

Topsy the Elephant after the electrocution

Satanizing AC was solely a business method. Edison knew there was nothing fundamentally wrong about AC but some quirks that could well be fixed. He could have adapted to the emerging business of DC – his company was the lead in electricity at that time after all – but he just didn’t want to let go his technology, and he was clearly willing to do anything to stop the competition. Now, if you look around you’ll noticed it didn’t turn out so well: AC supplies electricity to most modern houses and buildings, and we are even starting to send data through the electricity connections; moreover, even when AC is widespread, batteries supplying DC in small devices are still widely used too, so at the end both technologies found their market and coexist in our modern era.

Now, lately we have been reading all over the web about some new legislations that are designed to protect the business of companies that are threatened by piracy. The very fact that I’m using the word piracy is a byproduct of a whole media campaign that has been put around an emerging technology that has brought tons of changes to the world we know: the Internet. I’m, of course, talking about our friends ACTA, PIPA, SOPA, DRM, and any other acronym that threatens the web and the freedom of users.

The explosion of the Web has made millions of bytes readily available to each and every desktop around the world, and information can circle the entire planet in seconds, bringing little to no boundaries to the transmission of ideas and media. This is, of course, very good: it’s actually what Gutenberg once tried to accomplish using methods significantly slower. And now that we have achieved this level of communications, we have been convinced of the fact that using it is a crime.

Wake up! What companies like Sony, Microsoft and Apple are really trying to do is keep a market that they already lost before the technology. Sony had their time, when it was extremely difficult to let people listen to your music because the distribution media was expensive and required global logistics to reach every interested audience in the world. Nowadays, you can upload just about anything to Youtube (as crappy as it can be) and you will have a worldwide audience making you famous in a matter of days. Sony is, therefore, out of the equation these days, and alternatives like Creative Commons licenses are the modern way to deal with multimedia. That is because CC licenses start off by the (obvious) fact that everything can be shared in this modern world, and they focus on letting you control how to share your digital media.

Microsoft is supporting just about any idea to stop piracy. Of course! They don’t need it anymore. There were easy tricks back in the 90′s to stop the widespread usage of illegal copies of the MS Office suite, but they were intelligent enough to let it run in the wild, spreading proprietary formats and making adicts out of users that are left with no remedy but paying for licenses now that they are “serious” against piracy and have come with some very clever ways to stop illegal distribution of copies. If they were honest enough, they would step up and tell fellow companies “hey, this is just the way things work: people are going to share illegal copies, but you can still make business out of this as we did”.

And as fast as hardware technology grows, DRM is contaminating every invention out there, backed by companies like Apple that really think that if they are going to sink in their old fashioned business methods, the rest of the world must sink with them. We now have cables that are capable of blocking you from watching your own media, ala Terminator. If the transmission of bits and bytes are illegal, we could very well start pursuing people with any type of cables in the streets: LAN cables, USB cables, those are all dangerous potential criminals.

Finally, the one argument that has always looked stupid to me, is that old rhyme that says “Piracy money supports terrorist, ergo, all pirates are terrorists”. Heck, the truth is that, if flowers were better business, terrorists would be selling flowers, thus making flowers illegal, and florists would turn into terrorist – makes sense?

This is all very much like if back in the days when the commercial flights started to be popular, ship companies tried to make people believe that boarding airplanes was illegal. That just wouldn’t work and nowadays we have clear markets for both technologies and people even travel on cruise ships for pleasure. If any given company, as large as it can be, is not willing to adapt to a new technology, that has nothing to do with the general public. Creating laws to defend these dinosaurs, cuffing users for exercising their freedom to use technology,  is just insane. We have been led to believe that sharing copyrighted things is illegal, but little has been discussed about the nature of the laws that declare that illegal: do they even make sense in modern days?

So I’m clearly standing against these laws and treaties, but my cause is not to defend piracy: it’s recognizing this whole intelectual property system as obsolete. We should start over from scratch and think of a law system that can coexist with the technology we have in our hands now and only then can we speak about breaking the law.

Read More

Genkernel and AUFS for Writable Live Media in Gentoo

I recently wanted to create a Live USB with a very basic Gentoo installation but with the particular feature of being able to emerge packages from it. For that feature, I quickly learned I needed to have boot configured so that /usr was mounted read & write (or at least /usr/portage) and there are a number of ways to do that:

  1. Genkernel’s default scripts mount some of the common work folder into the memory (namely /etc, /root, /home and /var) and you can add /usr to that list by modifying the initrd.defaults included in the initrd. To do so, take a copy of the initrd.defaults script from /usr/share/genkernel/defaults, modify it to include /usr in the ROOT_TREES varible, then include that modified copy in the initrd by using commands similar to the ones included at the end of this post (there’s a subtle reason why you can’t just add /usr/portage but you can check that out by grepping ROOT_TREES in initrd.scripts). The obvious drawback of this approach is that, since these all are mounted on memory, you better have a lot of memory to handle the portage tree and the rest of your activities in the Live Media.
  2. Use UnionFS, which is supported by Genkernel. To do so, simply add the --unionfs to your call to genkernel and then add the unionfs flag to the kernel boot line in the boot loader (grub, lilo, syslinux, …). The drawback of this approach is that UnionFS is already being deprecated by AUFS so you probably want to keep reading for an AUFS solution.
  3. Use AUFS support in Genkernel, as explained below.

I’m not going to cover the basics of creating a Live Media from a Gentoo Stage 3 installation, there are plenty How-Tos about that. After you have emerged your kernel sources and genkernel, you need to emerge aufs3 (aufs2 if you are using any kernel before 3.0) and it will automatically patch your kernel sources to include aufs support (these brilliant Gentoo developers :) ) and only after that can you run genkernel to compile and install your kernel image. Yet, the module is compiled outside the kernel and not included in genkernel’s initrd by default, so it won’t be available while booting. To fix this, you can either use genkernel’s --all-ramdisk-modules, which will copy all of your kernel modules to the initrd and will probably bloat your initrd, or you can manually add the kernel module to your initrd. To do this, you need to decompress the initrd image, add the aufs kernel module and compress it back, as follows:

# mkdir initrd.fix
# cd initrd.fix
# zcat /boot/<your-initrd-image> | cpio -i
# cp -r /lib/modules/<your-kernel-version>/misc/ lib/modules/<your-kernel-version>/
# find . | cpio -H newc -o | gzip -c > /boot/<your-initrd-image>

After that, just add the aufs kernel option to your kernel line in the boot loader and you should be good to go! Share any questions or shortcuts in the comments.

Read More

The Kuest for Kalendars

It all started in September, a week before Software Freedom Day, when I was considering ways to leverage our events calendar in Floss-Pa to let everybody carry our calendar around. I started playing with remote iCals in my cell phone and, of course, in KDE which was at 4.7.1 at that time – and that didnt’ work. I noticed adding a remote URL as an iCal resource worked (as in “added the events in KOrganizer and the Calendar applet”) but caused constant Plasma crashes. I replicated the issue in both my PC and my Laptop, so I was quite sure it was a bug.

I had recently been accepted as a KDE Developer, just working on Kig mainly, so I thought it was a good chance to try patching somewhere else. I found out it was actually Akonadi related, so I started looking around for the particular KDE PIM project that dealt with the code and eventually (after fetching pretty much all of the main PIM projects) found the code at KDE PIM Libs and tried the simplest patch I could think of to avoid the crash: a simple if statement a couple of lines of code before the crash line that would basically “cancel everything if it didn’t look good”. The patch was rejected by Stephen Kelly, the author of most of the model code behind Akonadi, because the patch would hide a real and larger problem that was causing the issue.

I then knew I had to try harder to get a fix for that issue, so I thought about looking at KDE’s Bugzilla to see if someone else was already working on that. I remember I queried for “ical calendar” – which turned out to be a bad idea. I found a list of 4 bugs, none related to what I was looking for, but all of which I could confirm based on my couple of weeks of playing around with calendars in KDE.

There were two related to not being able to import custom timezones from an iCal resource. I had not tried that before, but Stephan Diestelhorst provided excellent feedback in the bug reports and attached an example of a resource with embedded custom timezones which made it trivially easy to reproduce the bug.  It turned out to be a bug in KDE PIM Libs, a project I had already checked out for working with the calendar issues. After two weeks of trying out examples and isolating the problem, I was finally able to submit a patch for review, which was eventually accepted and committed.

In the meantime, while waiting for reviews on my previous patch, I was still working on finding out what was the real cause of the Plasma crash, and at the same time working on another report of the four I found about the Calendar widget displaying events in the wrong time due to Timezone issues, although KOrganizer would display them right. That was one of the things I noticed the very first time I tried using iCal in KDE, so I was very interested in fixing it. After debugging the issue, and after a couple of weeks of finding out the right way to address the situation, I eventually came up with a patch that dealt with the inner mechanisms of Akonadi’s calendar support which wasn’t of the like of many: Sergio Martins and John Layt were in favor of an approach that would fix stuff upper in the layers of code, closer to the visualization layer. I had spent weeks working on that issue so  I was pretty sure there was no way to address it from the visualization layer, but I thought I might as well take another look at the visualization code just to make sure I was right… and I wasn’t. I found a much cleaner and better way to handle the issue and came up with a patch that followed John and Sergio’s suggestions and was eventually accepted and committed.

While working on the Calendar applet patch I was still using Stephan’s example from the previous bug I was working on, and while working with it in KOrganizer, I discovered the custom timezone caused KOrganizer to crash when editing the event. Since previously there was no way to include custom timezones in the iCal events, there was no way KDE users would notice the bug in the editors, so this issue had to be fixed together with the patch for custom timezones in iCal resources. Unfortunately, due to the review processes being independent and so close to the 4.7.3 release, the patch for custom timezones was included in 4.7.3 but the editor patch will have to wait for 4.7.4.

With most issues cleared out of the way, I was finally able to return to my original intention to fix the Plasma crash with remote iCals. I had previously posted in the Plasma Development mailing list about the situation I found was causing the crash, and Christian Mollekopf pointed me out to an interesting approach to achieve the same behavior without using broken code. I ended up implementing that approach which was recently approved and committed, so ending my long quest to fix many sorts of calendar issues in KDE.

The bottom line of this post is actually not to showcase my recent work in KDE Development, but to illustrate a whole process of collaboration among experienced KDE developers helping newcomers do real fixes and close bug reports – and how much can a user like Stephan help in the bug squashing process. Although some of those fixes are already included in 4.7.3, most of them are coming with 4.7.4 so I really hope the user experience with calendars in KDE improves in the upcoming release. There might have been loose ends here and there, so don’t hesitate to use the Bugzilla to let us know about them. I actually still have a list of checks and weird stuff I’ve seen along the way which are probably just more bugs to squash.

Read More

Sphinx and Jinja with Documentation on Gentoo

I, for reasons you surely don’t care about, have the doc flag in my defaults USE flags in Gentoo. That, in turns, helps me finding all kinds of bugs in ebuilds because these are usually not enough tested with doc flags. One of the oddest issues I’ve come across was emerging Sphinx and Jinja2 – and I just came across that issue again, so I thought posting something about it before I forget. The issue is that:

  • Sphinx requires Jinja for documentation
  • Jinja requires Sphinx

When using doc flag, that becomes a circular dependency, but that’s not a reason to report a bug. Instead, you should first emerge jinja without the doc flag, then emerge sphinx, and then emerge jinja with your regular flags:

# USE="-doc" emerge jinja
# emerge dev-python/sphinx
# emerge emerge jinja

Read More

All On Cross Domain Scripting

You probably arrived at this post because you were getting a return status of 0 when doing AJAX calls. That’s either because you are using the file scheme (you’d notice because your URL starts with file://) or because you are trying a cross domain call without noticing you need a whole lot of work to get that working right.

Although cross domain scripting is still considered a very risky thing to do, thus usually blocked by default in most modern browsers, there are plenty ways to work around these blocks. I’ll be posting about headers you can use to set cross scripting permissions, yet some other common methods are the crossdomain.xml file (which I don’t like much as it came up from cross domain scripting in Flash) and JavaScript based sockets like easyXDM.

First thing to know is that the reason you cannot access other domains from your AJAX call is because browsers enforce what is called the Same-Origin Policy for JavaScript. To work around that, the Cross-Origin Resource Sharing specification was created to define headers that can configure Cross-Domain permissions between browsers and servers.

Throughout the examples, we will assume you want to fetch a resource located at Domain B from a script located at Domain A. In order to get the most basic cross domain scripting working (that is, fetching a resource from another domain) you need to return a header from the resource at Domain B accepting Domain A as an allowed origin:

Access-Control-Allow-Origin: http://domain-a.com

once the browser receives that header, it will process the reply from the server and hand it to your script with status code 200, if successful. If you just wish to open that resource up for everyone to be able to fetch it from any domain, you can use a wildcard as the allowed origin:

Access-Control-Allow-Origin: *

Next complication level is trying to send cookies to that domain, e.g., for authentication. Your browser will not send those cookies by default unless you set your request to be a request with credentials:

var request = new XMLHttpRequest();
request.open(‘GET’, ‘http://domain-b.com/resource.xml’);
request.withCredentials = "true";

And even after you are sending credentials in your request, the server at Domain B must also allow credentials

Access-Control-Allow-Credentials: true

but a catch there is that, when allowing credentials, the origin must be specified explicitly (i.e., no wildcards allowed) or else no credentials are considered in the request, so you actually need to set up your response with these two headers:

Access-Control-Allow-Origin: http://domain-a.com
Access-Control-Allow-Credentials: true

so in case you want to open a resource for anybody to access it through scripts at their sites and also use credentials, you need to set the Access-Control-Allow-Origin header dynamically (see next section).

Access Control Through Dynamical Headers

One additional remark regarding Cross Domain Scripting with credentials is that  you may need to set headers dynamically per request. If the content is served from a dynamically generated resource (that’s a fancy name for a PHP script, for example) then all you need to do is use the request information to set the headers in the response. E.g., in plain PHP you would use the $_REQUEST variable and the header() function, or you could use the request and response objects of your preferred framework.

Another possible scenario is serving static content that is meant to be fetched by scripts in other domains, e.g., an image logo. Since you don’t want to allow cross site scripting for all pages served, you need to use tricks from the web server itself to set the response headers. If it’s an Apache server, for instance, you can use a combination of the mod_setenvif and mod_headers modules to generate headers depending on the requesting domain, for example:

SetEnvIf Request_URI “myimgage\.png$” CrossDomainResourceRequest
Header set Access-Control-Allow-Origin “*” env=CrossDomainResourceRequest

While there’s a lot more to read, these examples should get you going with your cross domain scripting… or you may already have thought of an alternate way to avoid this hassle!

Read More