Planet Lanedo

I meet Karl Isrich in a small restaurant. You maybe heard about the company he founded, MyVirtualTaylor, a pioneer of e-clothing. You would probably imagine Karl as one of those twenty-something golden boy. Instead, I face an average anxious guy, approximately forty years old with greyish hairs.

He asked me to go to this cheap restaurant because he could not afford a more expensive dinner. Lawyers, he said. When we sat down, he gave me a business card that used to be shiny six months ago. It simply says “MyVirtualTaylor, Isrich CEO”.

Hello Karl, thanks for the meeting. MyVirtualTaylor is an e-clothing company. But what is e-clothing exactly ?

Simply put, it’s 3D printing for clothes. We have developed a clothing printer that we sell and which is the size of a washing machine. Not being bigger than a washing machine was one of our top requirements before the launch.

The clothing printer has a tank of polymer, that you need to refill regularly, and seven dye tanks. We discovered that having seven primary colors was a good deal to reproduce most of the colors.

Through wifi, you send a .clo file to the printer then wait between ten minutes and one hour, depending on the size and the complexity of the model. Everything is automatic, you can even print a bunch of .clo in a row.

How do you get a .clo file?

We have an online editor on our website that allows you to design your own clothes. We have also some standard templates: shirts, ties, stuff like that.

In fact, when we launched, we didn’t really thought about that. We thought that there will be a new market for clothes creators. That’s why we wanted the .clo format to be open and documented. We sell the hardware but we didn’t want to enter the clothing market.

Can you really print anything? What are the limitations?

Currently, there are some constraints with the size. We have prototypes that can print as big as a king size bed sheet. But, of course, you can only print clothes made of polymer. No silk nor fabric.

Isn’t that a big limitation? After all, most of our clothes are made of fabric.

It should be noted that a lot of progress have been made with polymers. We can weave the polymer in a lot of different ways in order to have the properties we want.

But, most importantly, clothing material has always been about finding a compromise between style, comfort and durability. Durability being the critical point for quality clothes. The clothes have to go through hundred of washing cycles. Our solution was to remove durability from the equation.

Do you mean that printed clothes are not durable?

Not, they aren’t. But it is not the goal. Instead of cleaning them, you put them in the clothing printer and the polymer is cleaned, melted and ready to print new clothes.

Unfortunately, we still cannot extract the colors. The polymer is thus not perfect. We store the recycled polymer in a separate tank. When you print, you can allow the use of recycled polymer or not. It is good enough for every day but if you want a perfect white shirt for a wedding, you probably want the unused polymer.

The part of the polymer which is worn out goes with the waste to the sewers.

It sounds like an ecological disaster.

That’s exactly the rumor spread by our opponents.

But, while it is not perfect, you have to compare it with the traditional clothing industry. Clothes are usually made in huge factories in China, using harmful chemicals. Then, you have to take into account the transport, the storage, the shop. Not mentioning the gas needed to go to the shopping mall. To that, add the water and the soap used to wash the clothes. By contrast, we basically use electricity and reject very little polymer. With time, we hope to be able to recycle more and more.

Did you talk about opponents?

You know, I’m an engineer. I never really cared about anything but the technological aspects. When the first clothing printers were sold, people immediately started to exchange .clo files. They took there own clothes and make .clo files to be able to reproduce them.

One day, I received a letter from lawyers of the FCIAA, the Fashion & Clothing Industry Association of America. I’ve never heard of them before but, basically, they wanted me to stop my company because I was threatening their business.

I thought it was a joke. Really. At first I was like: ”Funny. It’s like the candle industry suing Edison for inventing the lightbulb”. But it’s not funny any more.

I can talk about this for hours. They are bad. Really bad. They are trying to destroy my life.

Can’t you let the lawyers handle that?

For the lawsuit, of course. But there’s a lot more. I’ve been contacted by politicians. They say that I’m destroying the economy. If my product works, there will be no shops for clothes hence no jobs. They asked me: “Do you know how many Americans are working in clothing shops?”. I was accused of being anti-patriotic. From nowhere, some news laws appeared saying that clothes should have a certification in order to save children from accidental suffocation.

From that point, it became immoral to print clothes. Last year, nobody ever thinked about printing clothes and, now, it is the worst thing after eating babies alive. There’s even webshops when you can order “Not Printed” labelled t-shirts. I’ve been attacked personally, investors have been turned down and, at the same time, I still need to pay expensive lawsuits.

Isn’t that true that it’s a threat for the economy?

It is a tool for making the life easier. Any invention which free people from unnecessary labor seems to be a threat to the economy. But if our economy is threatened by inventions that make life better for everyone, it’s the economy we need to change, not the inventions.

What will you do next?

I feel bitter. I’m an engineer with a new useful idea. And everything turns against me: big corporations, lawyers, politicians. Even random people in the street think that “It’s the guy destroying jobs and suffocating babies”. I’ve never signed for that. I’ve never been into politics or anything like that. Now, I’m thinking about settling somewhere in Europe but I’m afraid that the hand of the FCIAA will follow me there. 

Thanks Karl, I wish you the best.

Although I should keep a neutral standpoint, I’m feeling empathy toward this guy. As I’m packing my notes, I take a look at his clothes. “Are they printed?” “Of course” “Very nice. It’s impressive.” He sighs then try to smile at me: “Thanks. You will find the .clo on the Pirate Bay.”. His smile feels sad, despaired. We shake hands and he slowly walk away while I stay there, helpless.

 

This post is part of the Letters from the Future collection and is dedicated to Brokep for announcing his political involvement during the writing of this text. Picture by Anna Banana.

When debating, we usually consider that opinions are merely resulting of being exposed to logical arguments. And understanding them. If arguments are logical and understood, people will change their mind.

Anybody having been connected long enough on the internet knows that it never happens. Everybody stays on his own position. But why?

The reason is simple: changing opinion has a cost. A cost that we usually ignore. A good exercice is to try to evaluate this cost before any debate. For yourself and for the counterpart.

Let’s take a music fan that was convinced that piracy hurts artists. Convincing him that it’s not the case and that piracy is not immoral means to him that, firstly, he was dumb enough to be brainwashed by major companies and that, secondly, the money spent on CD is a complete waste.

Each time you will tell him “Piracy is not hurting artists and not immoral”, he will ear “You are stupid and you wasted money for years”.

This is quite a high cost but not impossible to overcome. It means that arguments should not only convince him, but also overcome that cost.

Worst: intuitively, we take the symmetry of costs for granted.

Let’s take the good old god debate.

For the atheist, the cost of being convinced is usually admitting being wrong. This is a non-negligible cost but sometimes possible. Most non-hardcore atheists are thus quite ready to be convinced. They enter any religious debate expecting the same mindset from the opponents.

But the opposite is not true. For a religious person, believing in god is
often a very important part of her life. In most case, this is something inherited from her parents. Some life choices have been made because of her belief. The person is often engaged in activities and societies related to her belief. It could be as far as being the core foundation of her social circles.

When you say “God doesn’t exist”, the religious will hear “You are stupid, your parents were liars, you wrecked your life and you have no reason to see your friends anymore”.

It looks like a joke, right? It isn’t. But, subconsciously, it is exactly what people feel and understand. No wonder that religious debates are so emotional.

Why do you think that some religious communities are fighting any individual atheist? Why do you think that any religion always try to get money or personal involvement from you? Because they want to increase the cost of not believing in them. Scammers understand that very well: they will ask you more and more money to increase the cost of you realizing it’s a scam.
Before any argument, any debate, ask everyone to answer sincerely to the question “what will happen if I’m convinced? What will I do? What will change in my life?”.

More often than not, changing opinion is simply not an option. Which settle any debate before the start.

And you? Which of your opinions are too costly to be changed? And what can you do to improve the situation?

 

Picture by r.nial.bradshaw

MBIM!

Winter is over, Spring is here, and along with the heat, rains and insects, ModemManager arrives with support for modems using the new MBIM protocol, available through the new cdc-mbim kernel driver in Linux >= 3.8 and a preliminary version of libmbim.

The current implementation provides:

• PIN unlock/change/enable/disable capabilities
• 3GPP modem support (GSM, HSPA, LTE…)
• Basic connectivity support

It doesn’t provide yet most of the side-features available in other modems (e.g. SMS messaging, location…), but it is a good start. If you don’t want to use MBIM just yet, check Bjørn’s list of issues and alternatives.

 

Where do I get it?

The ‘libmbim’ library can be found in the following Lanedo-maintained gitorious repository:

• git: https://gitorious.org/lanedo/libmbim

 
We also released a tarball in libqmi’s release place:
http://www.freedesktop.org/software/libqmi/libmbim-0.0.1.tar.xz

If you’re part of the freedesktop.org sitewranglers group, please check the still open libmbim project request.

The support is already in ModemManager git master, and will be enabled if libmbim is found during configure. You can also explicitly request to use MBIM support with the new –with-mbim configure switch.

 

Thanks

Special thanks go to Lanedo for sponsoring the initial libmbim development, Bjørn Mork for his suggestions and tests with multiple MBIM-capable modems, and Huawei for sending me a E367 with MBIM firmware.

Filed under: Development, FreeDesktop Planet, GNOME Planet, Lanedo Planet, Planets Tagged: freedesktop, libmbim, MBIM, ModemManager

During most of my life, I thought that there was only two ways to give money to someone. Firstly, in exchange of something you want/need but not available without paying. It is called “buying”. The other occasion is giving money for nothing, because you want to help someone in a bad situation and want to feel good about it. It is called “charity”.

If I forget about gifts, which are a rare exception happening only within my close social circle, every transaction is either a purchase or a charity donation. Nothing else.

The implications are huge. It means that something has one and only one fixed price, fixed by the market and identical for everybody. That price is perceived as the real value. People will pay an expensive ticket for a violin concerto but will not pay attention if the same artist plays in the metro. It is free, thus worthless. In our society, price and value are synonym.

In that world, when you plan to earn money, you only have two solutions: pledge for charity or give something that customers cannot get for free. On the internet, where nearly everything can be found freely, this translated into two business models: a paywall (your readers being your customers) or advertising (your readers being the product you sell to your customers). The paywall proved to be ineffective (because people can get what they want for free anyway) and the advertisements proved to be very lucrative for the intermediaries (like Google) but not for the content providers. It also has the result of making the content providers caring more about advertiser’s interests (their customers) and to think about their followers (their product) only in terms of volume.

When I joined Flattr, in 2010, I thought it was only a way for people to give me charity. My bet was to invest 24€ in a year in order to earn more. I told myself that I would quit Flattr if made less than the, at the time mandatory, 2€/month.

I’ve earned more but, most importantly, learned much much more.

I discovered that transactions are not only of the “buying because you have no choice” or “charity” kind. It could be something hybrid, something I call the “free price”.

I thought that what I wrote on my blog had no value. My writing would be valued only if published in a book. But I discovered that it nevertheless had value for some. A different one for each reader. Not fixed by the invisible hand of the market but by their personal history. Flattr allowed people to pay for each of my blog post according to the value they saw in it. It is not charity, it is not giving. It is “paying freely”.

In a sense, it is a lot more fair. A poor reader will be able to give me 0.10€ while a richer reader can give me 1, 10 or even 50€. The content producer dilemma was “publish something for free and make it worthless in order to reach a wide audience” or “keep it confidential to earn money and monetize the content”. Now, with the “free price”, you can have both. Making stuff for free while keeping an high value.

As Amanda Palmer said, this is not new. It always existed for street artists, for waiter’s tips. But I was confusing it with charity and may not be alone in that case. It is clearly not charity. You pay for something, for a product. It is a free price.

This has the groundbreaking effect of putting into question the traditional equation price = value. Because there’s not one fixed price but as many prices as customers.

While I’m very excited about this, I also realise that this is the main weakness of Flattr: it is too disruptive.

Flattr will only work for people already convinced that this third transaction model is possible. It will only work for the people that are already seeing value in stuff that have no price. People that are ready to go through the hassle of creating an account, sending money, etc.

But what if we could transform Flattr into an educational tool? Teaching people the joy of paying for stuff without a fixed price? After all, it’s exactly the effect it had on me.

What about a Flattr paywall as a Trojan horse? A Flattr paywall is something I already explored in my story “The Publisher’s Dilemma”. To access a list of content, a Flattr account would be required and any content you access would be automatically Flattered. Psychologically, content producers will then learn to make content for a “free price” without being required to publish completely for free (which is, thanks to the industry lobbying, something artists are afraid of).

On the other hand, many consumers who never bothered to pay for something may think “Hey, I can access many content on many websites for only 2€/month. Let’s create an account.” Once their account is credited, they may Flattr other content. After all it doesn’t cost them more money. And, like I did myself, find themselves increasing their monthly Flattr.

Flattr is an awesome tool for people who believe in a “free price”. But it could go one step further and become an advocacy tool for the “free price”. Something which is shaking one of the deepest foundation of our society, the infamous price tyranny.

 

Picture by FrostWire.

We have entered in an incredible world of free content. If it can be copied, you can find it for free on the Internet. It is as simple as that and it is awesome. We never dared to dream about such a world where every content, every knowledge was free, shareable with everybody. But it’s the world we are living in.

Nevertheless, on the individual scale, this evolution is understandably not accepted by the people who were used to sell content by purposedly confusing a given content and its physical medium. After a failed attempt to artificially make content impossible to copy, some tried to monetize their content through advertisements. The fundamental flaw in that model being that it would keep the money in the real big-blue-ceiling world. A world which, unlike the virtual world, is physically limited. The growing virtual world would become nothing but a huge billboard for the real world. Not realistic…

When you cannot block people to access stuff for free but want to make them pay anyway, you have only one solution left: morality.

Current calls to morality are incredibly negative: “Not paying is bad”, “If you don’t pay, content creators will die”, “If you don’t pay, you will get sued”, “Not paying is stealing”, …

It leads to a business model based on fear and guilt. A world where everyone has to pay the same price before consuming the content. Not to mention the inherent contradiction of wanting to see the content spreading while, at the same time, blocking some from accessing it.

But what if the call to morality was actually positive? “You don’t have to pay but it will be appreciated”. “If you pay, I will be able to create more content”. “If you can’t afford to pay for it, at least share it with your friends, spread it!”.

In that new virtual world, only those who liked the content would pay. And they would pay the amount they want. Does it seems unrealistic because most people would choose to not pay? But it already exists. A lot of waiter and waitress in the world actually earn a living from tips. Or street artists like Amanda Palmer in her early days. Those tips are optional and paid afterwards. The amount being proportional to the quality of the service or the pleasure we had. Why is it working? Because we are used to that system. Because we are positively compelled to give a tip. Because we can give what we find reasonable for our budget.

In order for this system to work in the virtual world, it should be incredibly easy to give a tip without even thinking about it. Yet, such a system already exists. It is called Flattr and I already gave you a presentation. The strength of Flattr is that you pay in advance a monthly tip. There’s no way to get over your budget as it is monthly fixed.

But this would be even more awesome it the tip could be automatized. We spend a lot of time liking picture on Instagram, video on Vimeo, favouriting tweets and listenning to song on Grooveshark. Guess what? Flattr dit it! Starting yesterday, those like/favourite/recommend actions will automatically give a Flattr to the authors (if you enable it, of course).

And this service is open to any content provider. Each time you would like something on your favourite platform, you would send a tip. 90% of that tip goes directly to the author, 5% goes to Flattr and 5% goes to Medium.

For sure, those tips might appear negligible in the first time. But, as a content creator, isn’t it compelling to earn money because people wanted to give you money? Not because they were deceived into buying the entrance ticket but because they actually enjoyed your content? Wouldn’t it send a positive signal to new generation of content creators?

Feel free to consume the content. Feel free to share it. Feel free to tip it.

Just feel free…

 

Disclaimer: I earn between 4€ and 110€ per month with Flattr. I’m not affiliated with Flattr in any way but I’m really excited by the philosophical implications of Flattr. That’s why I’m writing so much about it. I would of course welcome any similar service, especially if it could be decentralized. This post was first written on Medium. Picture by Parisa.

Who needs borders and local regulations when you have internet? The answer to this question is why I like Bitcoin so much. It is a trully decentralized currency.

But Bitcoin has some major issues, something which was covered by Rick Falkvinge on his blog.

The first problem that hits any Bitcoin user is usability. Using Bitcoin is complicated and cumbersome. You can’t send a simple payment with a comment to someone without asking him first to create a dedicated Bitcoin address. You have to care about making a secure backup of your wallet. This makes things nearly impossible to use for a huge percentage of the population.

The second problem is trust: how do you know that the Bitcoin address is valid and was not replaced through a man-in-the-middle attack? How do you know that you will well receive the goods or the service? Sure, many escrow services appeared but they make a Bitcoin transaction even more cumbersome.

The third problem is the power of the exchanges. The whole Bitcoin economy runs with only a handful of Bitcoin exchange services, MtGox being the bigger. This de-facto centralisation is a big weakness for Bitcoin. MtGox has the power to control the price. If MtGox has any problem, the whole value plummets.

Last but not least, people have to trust bitcoins. Except when Internet Archives offers its employees to receive a Bitcoin salary, most services accepting bitcoins are in fact converting them immediately to dollars/euros. It means that Bitcoin is seen as a transport, not a currency.

I even drafted a proposed solution as a Bitcoin-banking decentralized protocol. Recently a quite old project called Ripple surfaced and brought the idea to a whole new level: what if every one of us was a bank and we decide who we trust and at what level.

Remember when you go on a trip with a bunch of friends. Everybody pays for some stuff and, at the end, you try to equilibrate the balance. It was a nightmare until you discovered Tricount. Well, Ripple is basically a decentralised Tricount at the scale of the internet. We are 7 billions friends on the same trip. We pay for each other, we owe some people money and that’s it.

The beauty of it is that it solves all Bitcoin’s hurdles as long as there are enough people in the network. It is easy, decentralized. It will also make money exchanges completely obsoletes. It has the potential to create a true P2P economy.

Now, it is only a proof of concept. Firstly, if the client is opensource, the server is not (yet). And that’s a problem because there’s no competition to ripple.com at the moment. It means it is hugely centralised.

Also, you still have to exchange weird addresses like rKXFsg5EuG4BzLxdTBFXJq2a6iNfyx1hRX (this is my actual Ripple address). In order to become popular, Ripple should allow you to directly connect with your Facebook/G+/Twitter friends so you can trust them or send them money. After creating the Ripple wallet, the process is still very mysterious.

Ripple also raises a few questions. Is its own internal money (XRP) making Bitcoin obsolete? Or is Bitcoin going to stay? And what about the Dollar or the Euro? At least, an interesting experiment to follow.

 

Picture by Lee Haywood

 

Last week, Xamarin announced their Xamarin 2.0 product, which contains Xamarin Studio, a brand new version of MonoDevelop. Miguel describes the impressive amount of work that went into Xamarin Studio in his blog post The Making of Xamarin Studio. Xamarin Studio is written in C# and for a large part relies on GTK+ 2.x to make the product cross platform, shipping versions for Mac OS X and Windows, but Xamarin Studio should also run just fine on Linux. As an addition to Miguel’s blog post, I will describe some of the interesting work that we have done under the hood in our ongoing collaboration with Xamarin.

At Lanedo, we have been working with Xamarin for the last 1,5 years to improve the GTK+ stack on the Mac OS X and Windows platforms. This work was based on the GTK+ 2.x stack, because MonoDevelop is a GTK+ 2.x application. We have made an effort to upstream all fixes and improvements that were suitable to be committed in the upstream projects. Where possible, fixes have also been merged into the GTK+ 3.x branch. Furthermore, we have helped with the implementation of the gorgeous GTK+ theme (available in the xamarin-gtk-theme module on GitHub) that is used in the product and makes Xamarin Studio look like a native application, and not look completely out of place on Mac OS X or Windows like GTK+ applications typically do. In my opinion, the approach that Xamarin has taken for theming will be useful for many other GTK+ applications running on Mac OS X as well.

Most of the work we have done was under the hood, but that does not make it less interesting. Some of the things we have worked on are:

• Many, many annoying interaction problems and crasher bugs with GTK+ on Mac OS X have been fixed. Such as non-working tree view expanders, broken key navigation, issues with window resizing, and gaps in clipboard support.
• Keyboard fixes for Mac OS X: everything with respect to keyboard modifiers and key mapping has been reviewed, several problems with typing dead accents, keyboard accelerators, etc., have been resolved.

The GTK+ "testtext" test rendering glyphs from different scripts using CoreText, but through font fallback done in Pango itself.

• Xamarin sponsored the work to properly implement font fallbacks in Pango’s CoreText backend. As a result, text involving glyphs from different scripts will now render beautifully on Mac OS X.

gtk-demo with overlaid scrollbars. Though, for full admiration you really have to see it in person!

• We have worked very hard to make scrolling behave as native as possible on Mac OS X. Support for representing "precise deltas" from NSEvents in GdkEvents has been written, which already gives you much smoother scrolling on Apple hardware and support for momentum events (built-in kinetic scrolling).

  The next step was to have overlaid scrollbars and rubber banding, like native applications on Mac OS X Lion and higher. To achieve this, we actually place a CALayer on top of the NSView in which the GTK+ application is drawn, and draw the scroll bars in this layer. We have tweaked the event handling so that information about the gesture phase is considered, in order to be able to implement rubber banding. Unfortunately, this code is not in a state such that it can be upstreamed.

  Performance problems were uncovered in how GDK handled scrolling on OS X, apparently, much more of the widgets was being redrawn than necessary. These problems have been fixed and upstreamed, and scrolling is now pretty smooth.

Cocoa controls embedded in a GTK+ application using GtkNSView, the web page is shown in a native WebView.

• Work has been done on a GtkNSView widget, which allows native Cocoa widgets to be embedded in a GTK+ application. For example, it is possible to embed a native WebKit control in a GTK+ application. Work on this widget is still ongoing in Bug 619155.
• Some improvements were made so that GTK+ applications render at higher resolution on Retina displays. This includes work to render stock icons at a higher resolution.
• A few Windows problems have been resolved, including a crash with Windows IME, font substitution and other minor problems.

We have very much enjoyed working with Xamarin over the last 1,5 years and are grateful for the large contributions they have made to the involved upstream projects. In my opinion, the GTK+ 2.24 branch is these days in a very good shape for cross platform application development for the Linux, Windows and Mac OS X platforms. The work that has been done has immediately been beneficial for other applications, for example the native GIMP application for Mac OS X has been much improved. These days it is available as a proper application bundle in a DMG installer (due to the work by Clayton Walker) from the main GIMP download page and it works very well!

We look forward to the continued collaboration with Xamarin to further improve the GTK+ stack for use by Xamarin Studio!

When I explain the tips I use to stay at inbox 0, a common reaction is to pretend to do the same with unread emails, read emails being “archived”.

Given how the human brain works, this is unfortunately not true.

Firstly, we don’t deal with individual items as we are with groups. Meet 3 persons? Those are Alice, Bob and Charles. Meet 15 persons? Those are a group. Or a lot of people. Same applies for your inbox: if there are hundreds of emails in your inbox, your brain doesn’t care about the read/unread status. It just consider it as “a lot of emails”.

The natural consequence is that you will have the tendency to not react to a new email. You have no reward for dealing with an email: 100 or 101 doesn’t change anything. Worst: dealing with an email looks useless and pointless. It will not change anything. While, if your inbox is empty, you will have a natural tendency to act as quickly as possible. Seeing your inbox empty is a relief, a reward in itself.

The second point is that the more time you are given to do a task, the harder the task will look. If I tell you that you have one hour to write an essay about a subject, you will rush and do it. If I give you six months, you will first procrastinate then do some research then realize that you have only one month left to do what is supposed to be a six months work. This will make you feel that the task is really hard. An email sitting in your inbox is doing exactly that: reminding you that you have a task that started the day you received an email. The longer the email stays in your inbox, the harder the task will unconsciously look, the more you will tend to procrastinate.

Of course, that doesn’t mean it’s impossible to be very efficient without an inbox 0. It is just a lot harder and requires more energy.

Now, maybe it’s time to ask the opposite question: why are you not at inbox 0? Why is that given email still in your inbox?

1. Because you need to reply to that email

In that case, don’t hesitate to go for the quick reply. A fast and quick reply is often better than a deep reply, one month later. If your reply really needs an in-depth investigation, it’s not a simple reply any more, it’s a task in itself and should be in your todo-list. Once replied, archive the mail immediately.

Try to answer as soon as you are reading the mail. But this does not mean you have to check your email every five minutes. Disable notifications and choose when to read your emails.

2. Because you need to do something about this email

Then take your todo-list and write that something. A todo should always start with a verb. It’s an action. The problem with an email is that, very often, you don’t know exactly what to do. You know you have something to do but you procrastinate because the next action is not clear.

Sit down, write the action you have to take regarding this email then archive the email.

A good todo-list will not bother you with every possible task every day but only give you what you can achieve, avoiding the constant exposition leading to procrastination.

3. Because the information in the email might be useful

If it is an information you need for another task, copy that information in your todo list, your agenda or where it belongs. Then archive the email.

If you don’t need the information immediately, archive the email. Your archive is not a trash, you can find the information you need at any time there.

4. Because you are not sure about what to do

Sometimes, you have to take decision. Such emails are often related to an invitation: I should reply but I still don’t know if I’m going. Action is simple: take your decision, put the event in your calendar.

If you are really unsure, put it in your calendar anyway and tell the sender that you have the event in your agenda but are not 100% sure to attend. Then archive the mail.

If the decision is really hard, just make it a task in your todo list: “make a decision about X”. Don’t let a related email clutter your inbox.

5. Because you are not sure if you should keep this email or not

Archive it. As I told you, forget the trash, archive everything.

6. Because you are not sure in which folder you should put this mail

Don’t do folders. Don’t. Archive. That’s exactly the main reason to avoid folders.

7. Because your inbox is your todo-list

While this is not impossible, it requires a lot more energy. It also makes your todo list completely weird: in most case, an email is not a task. It is related to a task, which is something completely different. Worst: an email called “weekly report” could contain multiple tasks and the title say nothing about them.

Maybe it’s time to hunt for a good todo-list. But, surely, your inbox is the worst possible solution.

Conclusion

When a mail stays for more than a few days in your inbox, you have to be conscious about the fact and not accept it with any excuse like “I’m already efficient enough”. You would never leave all your letters from the past years in your front-yard mailbox. Same applies for your inbox.

Having an inbox filled is not a fatality. You can, today, claims control over your inbox simply by realising why a mail sometimes rots in your inbox and by taking some proactive measures.

You would be surprised how email can still be very efficient and easy.

 

Picture by John Morgan

Recently Carlos added FTS4 and snippet support to Tracker. We merged that to master after doing some tests and have reduced the database size on disk by doing this. I released 0.15.2 yesterday with the FTS4 work, and today I decided to add a richer experience to tracker-search.

Below you can see me searching for passport and sue found in some of the documents indexed on my machine. The colour there is quite nice to separate hits and snippets/contexts where the terms were found. This search without any arguments really will search ALL resources in the database:

This second screenshot shows searching for love with all music in particular. So you can use this for all areas of tracker-search:

With any luck, we will be releasing a 0.16.0 in time for the next GNOME release with this all available in!

So I wanted to have ModemManager in OpenWRT to handle my mobile broadband connections.

OpenWRT has its own built-in support to handle mobile broadband modems of different types, mainly for AT-controlled modems, but also for QMI-controlled modems through libqmi and qmicli (now available in upstream OpenWRT). But hell, why not package ModemManager for that work, and provide a unified single DBus API to control any kind of mobile broadband modem in the same way?

ModemManager itself isn’t a very complex daemon, so it doesn’t have many dependencies, just: glib/gio, dbus, udev/gudev and libqmi. In the world of OpenWRT, though, none of these are blessed dependencies, as they have a lightweight replacement for everything: libubox (instead of glib), ubus (instead of dbus) and hotplug2 (instead of udev). These replacements target minimal embedded systems, and are perfectly integrated into OpenWRT and provide the building blocks of other core software like the netifd network interface daemon.

Still, having ModemManager available as a OpenWRT package seems like a good idea if you’re not very constrained by the hardware you’re going to use. If you are building a system which should support multiple broadband modems from different vendors, or just building a base system for one specific modem model but leaving the door open to change the model in the future, ModemManager seems a good choice.

 

systemd’s udev in OpenWRT

Since April 2012, udev sources are now integrated within the systemd source code. And that means that udev packages in OpenWRT would need to get updated in order to get built from that new repository. systemd provides some hints on what would be the best way of creating a minimal systemd build which can be used to gather just the udev-specific libraries and binaries. The easiest way involves compiling the whole systemd package with all optional features disabled, and that would get us not only systemd (which we won’t need), but also udevd and libudev. Easy, yes, but then we have some issues in the environment that we would need to handle…

In OpenWRT, the udev package is available in the trunk repository, where only the core OpenWRT packages are given. But in order to get the new udev compiled out of systemd we end up needing additional dependencies (dbus, libcap) which are only available in the packages repository. Moreover, if you also want to compile the glib-based gudev library, glib is also needed, which is also only available in the packages repository. And you cannot require a package from trunk to depend on a package from packages, so… ended up creating a new systemd-udev package within the packages repository.

As I was targeting the uClibc toolchain, I also ended up requiring some additional patches both in uClibc (e.g. to enable utmpx) and systemd to get the thing properly compiled. The patches applied in the systemd sources are just to handle missing features in uClibc (execvpe, secure_getenv…). Whenever uClibc includes those, we’ll be able to remove those patches.

 

ModemManager in OpenWRT

I prepared an easy setup of ModemManager packaging in OpenWRT, where all possible plugins get compiled. It wouldn’t have been difficult to allow selecting via configuration which plugins to get compiled, but didn’t want to spend time on that.

I sent the patches to the OpenWRT mailing list already, but if you want to give it a quick try, you can just clone the Lanedo OpenWRT repositories (trunk and packages) from gitorious.org:

$> git clone git://gitorious.org/lanedo/openwrt.git
$> git clone git://gitorious.org/lanedo/openwrt-packages.git


You’ll want both ‘modemmanager-support‘ branches in those git repositories.

ModemManager will be available under the ‘Network’ section in the OpenWRT build configuration; but only after explicitly selecting the following kernel modules (all under Kernel Modules/USB support):

• kmod-usb-net
• kmod-usb-serial
• kmod-usb-acm

Once all above are enabled, ModemManager can be selected, and it will itself pull all its other required dependencies (systemd-udev, glib, dbus, libqmi…).

 

No LuCI? No connection manager?

Well, not yet . If you want to use ModemManager you still need to have your own connection manager software to manage the connectivity. ModemManager will get your mobile broadband modem connected, but you still need to either call pppd or setup the wwan interface yourself (depending on the data port type). This is really not a big deal; and you can just setup a shell script using the provided mmcli command line interface to talk to ModemManager.

Filed under: FreeDesktop Planet, Lanedo Planet, Packaging, Planets Tagged: libqmi, ModemManager, openwrt, systemd

About

Determine candidates and delete from a set of directories containing aging backups.
As a follow up to the release of sayebackup.sh last December, here’s a complimentary tool we’re using at Lanedo. Suppose a number of backup directories have piled up after a while, using sayebackup.sh or any other tool that creates time stamped file names:

 drwxrwxr-x etc-2010-02-02-06:06:01-snap
 drwxrwxr-x etc-2011-07-07-06:06:01-snap
 drwxrwxr-x etc-2011-07-07-12:45:53-snap
 drwxrwxr-x etc-2012-12-28-06:06:01-snap
 drwxrwxr-x etc-2013-02-02-06:06:01-snap
 lrwxrwxrwx etc-current -> etc-2012-12-28-06:06:01-snap

Which file should be deleted once the backup device starts to fill up?
Sayepurge parses the timestamps from the names of this set of backup directories, computes the time deltas, and determines good deletion candidates so that backups are spaced out over time most evenly. The exact behavior can be tuned by specifying the number of recent files to guard against deletion (-g), the number of historic backups to keep around (-k) and the maximum number of deletions for any given run (-d). In the above set of files, the two backups from 2011-07-07 are only 6h apart, so they make good purging candidates, example:

 $ sayepurge.sh -o etc -g 1 -k 3 
 Ignore: ./etc-2013-02-02-06:06:01-snap
 Purge:  ./etc-2011-07-07-06:06:01-snap
 Keep:   ./etc-2012-12-28-06:06:01-snap
 Keep:   ./etc-2011-07-07-12:45:53-snap
 Keep:   ./etc-2010-02-02-06:06:01-snap

For day to day use, it makes sense to use both tools combined e.g. via crontab. Here’s a sample command to perform daily backups of /etc/ and then keep 6 directories worth of daily backups stored in a toplevel directory for backups:

 /bin/sayebackup.sh -q -C /backups/ -o etc /etc/ && /bin/sayepurge.sh -q -o etc -g 3 -k 3

Let me know in the comments what mechanisms you are using to purge aging backups!

Resources

The GitHub release tag is here: backups-0.0.2
Script URL for direct downloads: sayepurge.sh

Usage

Usage: sayepurge.sh [options] sources...
OPTIONS:
  --inc         merge incremental backups
  -g <nguarded> recent files to guard (8)
  -k <nkeeps>   non-recent to keep (8)
  -d <maxdelet> maximum number of deletions
  -C <dir>      backup directory
  -o <prefix>   output directory name (default: 'bak')
  -q, --quiet   suppress progress information
  --fake        only simulate deletions or merges
  -L            list all backup files with delta times
DESCRIPTION:
  Delete candidates from a set of aging backups to spread backups most evenly
  over time, based on time stamps embedded in directory names.
  Backups older than <nguarded> are purged, so that only <nkeeps> backups
  remain. In other words, the number of backups is reduced to <nguarded>
  + <nkeeps>, where <nguarded> are the most recent backups.
  The puring logic will always pick the backup with the shortest time
  distance to other backups. Thus, the number of <nkeeps> remaining
  backups is most evenly distributed across the total time period within
  which backups have been created.
  Purging of incremental backups happens via merging of newly created
  files into the backups predecessor. Thus merged incrementals may
  contain newly created files from after the incremental backups creation
  time, but the function of reverse incremental backups is fully
  preserved. Merged incrementals use a different file name ending (-xinc).

See Also

Sayebackup.sh – deduplicating backups with rsync

Linux kernel 3.8 comes with a new ‘cdc-mbim‘ driver… let’s see what all this is about…

 

What is MBIM?

The Mobile Interface Broadband Model (MBIM) is a new standard developed by the USB Implementers Forum, specifically designed for high speed mobile broadband modem devices.

This new USB networking subclass defines two separate new features:

• A new MBIM USB device model, providing multiple IP connections over a single USB interface, and without the need of 802.3 frames (as was the case with ECM and NCM)
• A new MBIM control protocol to talk to modem devices

 

MBIM message types

The protocol defines different message types with different formats.

Some of them are used to establish the channel of communication with the modem:

• Open Message (Host->Modem): Initialization request.
• Open Done Message (Host<-Modem): Initialization response.
• Close Message (Host->Modem): Close request.
• Close Done Message (Host<-Modem): Close response.

Some of the messages are used to report errors in the protocol; which may be sent either from the host or from the modem:

• Host Error Message (Host->Modem): Host-reported error.
• Modem Error Message (Host<-Modem): Modem-reported error.

And finally, some messages provide access to the different CIDs (Command IDs) defined in each Service.

• Command Message (Host->Modem): Request of a given command.
• Command Done Message (Host<-Modem): Response to a given command.
• Indication Message (Host<-Modem): Unsolicited messages sent by the modem.

These last three messages support protocol-defined fragmentation. If the sender of the message finds out that the message is longer than the maximum control transfer size defined when the communication channel was opened (with an Open Message), it will be able to split it into sorted chunks and send them one by one over the wire. Each of these fragments specify the total amount of fragments expected, as well as the order in the sequence. The receiver of the fragments will therefore need to wait for all fragments to arrive – which should arrive in order – before processing the message.

 

MBIM services

The protocol defines a basic set of different Services:

• Basic Connect: Which provides the support for basic IP connectivity
• SMS: SMS messaging
• USSD: Unstructured Supplementary Service Data
• Phonebook: Handling contacts and such
• STK: SIM toolkit
• Authentication
• Device Service Stream

Only the Basic Connect one is mandatory in every MBIM device; others are optional. Vendors or even Network Operators can also extend the functionality of the device with other services; for example to support other protocols embedded within MBIM (e.g. QMI or AT within MBIM); or just to provide specific new features.

 

MBIM commands

For each service, MBIM defines a set of Commands (CIDs); and each command can then be divided into 3 actions:

• “Set“: An user-requested action to change the modem state or configuration
• “Query“: An user-requested action to query the modem state or configuration
• “Notification“: An unsolicited report of the modem state or configuration

Not every action is supported by every command. For example, the “Device Caps” command in the “Basic Connect” service only supports the “Query” action, while the “Radio State” command of the same service supports all “Set“, “Query” and “Notification“.

But how does this match with the message types defined before?

• Host-created “Set” and “Query” requests are sent using “Command Messages“.
• Modem-created “Set” and “Query” responses are sent using “Command Done Messages“.
• Modem-created “Notifications” are sent using “Indication Messages“.

 

MBIM basic types

The protocol not only defines which action is supported in each command; it also defines the contents of the request, response and indication messages. The contents of each message are composed of collections of basic types defined by the protocol:

• Unsigned 32bit integers: Even for the most simple values (e.g. booleans), little-endian 32 bit unsigned integers are used.
• Strings: UTF-16LE encoded strings are always used.
• UUIDs: The protocol defines a special 16-byte-long UUID type, e.g. to define Service IDs
• Arrays: Collection of N values of a given basic type.
• Structs: Sequence of other basic types, given in a specific order.

As you can see, there is not much effort (none, actually) into making the protocol efficient in terms of overhead introduced; more than half of the bytes in each message will very likely end up being NUL bytes. Also, it is assured that each field within a message is aligned in a 32bit boundary, which makes it easier to read independent fields directly from the binary stream.

 

How does this compare to QMI?

There are quite some differences between this protocol and QMI. To name a few:

• Less basic types in MBIM, specially regarding signed/unsigned integers. QMI defines many more integer types, of different sizes and with different sign.
• Basic strings are given in ASCII in QMI
• Struct types are re-used in MBIM, as opposed to QMI, where each TLV would define the struct contents as it needed.
• There is no ‘client allocation’ needed in MBIM, i.e. the process opening the MBIM port doesn’t need to allocate clients for the different services.
• Fragmentation is built-in in the MBIM protocol; QMI required special TLVs and logic to handle messages that may be longer than the standard message size (e.g. when transferring the PRL list).
• Binary representation of the message is completely different. In QMI you would have a list of TLVs, one after the other. In MBIM, the contents of the message are split into two different sections: one which contains the fixed-size values and one for the variable-size values. For example, a string in MBIM is defined by two values in the fixed-size region (offset and size), while the real string data is within the variable-size section.

 

libmbim

The ‘libmbim‘ library is an attempt to write a protocol support library, as previously done with ‘libqmi‘.

The current codebase, GLib/GObject/GIO based, is pretty similar to what libqmi provided, with a ‘MbimDevice‘ GObject to handle the communication through the /dev/cdc-wdm port, as well as ‘MbimMessage‘ types to handle the creation of commands. Given that there is no need for client allocations in MBIM, there is no ‘MbimClient‘ object needed. Also, given that the contents of each message are pre-defined, it wasn’t considered the need of input and output ‘bundles’ to handle collections of TLVs, as done in libqmi.

Fragmentation of incoming/outgoing messages is handled internally by the ‘MbimDevice‘, which is a nice thing to have, as it simplifies the usage of the API. The user just creates a message, as long as it needs it to be, and the ‘MbimDevice‘ will take care of splitting it into fragments. When receiving a message, the user will also receive already the full message, once all fragments have been combined internally.

The messages and the services are defined in a JSON dictionary, and as with libqmi, all the message handling code is auto-generated from there. The capabilities of the mbim-codegen are still not in pair with those in qmi-codegen yet, though. The generator doesn’t support e.g. “Set” requests with parameters, or “Indication” messages. As previously said, this currently is a proof-of-concept, someday we’ll support all those properly.

 

mbimcli

The project comes with a command line utility (mbimcli), which allows (will allow) running “Get” or “Set” commands directly from the shell. There aren’t many supported commands yet, so I cannot show many more examples than this one:


$ sudo mbimcli \
    -d /dev/cdc-wdm1 \
    --basic-connect-query-subscriber-ready-status

[/dev/cdc-wdm1] Subscriber ready status retrieved:
      Ready state: 'device-locked'
    Subscriber ID: 'unknown'
        SIM ICCID: '984310311520086950F1'
       Ready info: 'unknown'
Telephone numbers: ''


Hint: Using --verbose will show you the raw binary message contents!

 

Where do I get it?

The ‘libmbim’ library and the ‘mbimcli’ utility can be found in the following Lanedo-maintained gitorious repository:

• git: https://gitorious.org/lanedo/libmbim

 
There is already a request to host this project in the freedesktop.org infrastructure, so it may end up getting moved there at some point.

 

Help! Contribute! Sponsor!

This development was kindly sponsored by Lanedo GmbH. If you want to help, or sponsor further development in libmbim, mbimcli or the ModemManager integration, just let us know!

Filed under: FreeDesktop Planet, GNOME Planet, Lanedo Planet, Planets Tagged: freedesktop, libmbim, libqmi, MBIM, ModemManager, QMI

After quite some time without touching Tracker code, last week I finally could get back to a branch that’s been sitting there for some time now. On fts4, sqlite requirements have been updated to >=3.7.9 so we can stop compiling our custom FTS module and start using what comes with libsqlite.

What does this mean? Internally there’s less code on our plate (and non-stale), and external content support in FTS tables brings us no performance nor file size decreases. plus we get all recent hot improvements in sqlite releases for free.

A bit more on the user point of view, a feature that became possible with this swich is the support for fts:snippet(), which you can use in SparQL queries to get snippets of the matched text:


$ tracker-sparql -q "select nie:url(?file) fts:snippet(?file) fts:rank(?file) where { ?file a nfo:Document ; fts:match 'reference' } order by desc (fts:rank(?file)) limit 3"
Results:
file:///home/carlos/Documents/Papers/pdf_reference_1-7.pdf, ...Reference Streams G8.1872911 Cross-Reference..., 46.0
file:///home/carlos/Downloads/addison.pdf, ...GLU are described in the OpenGL
Reference Manual. The more useful GLU..., 40.0
file:///home/carlos/Downloads/ThesisHo.pdf, ...A8 ]+ ) is also included for
reference. In the third experiment, we apply..., 40.0


So its easier to the eye, tracker-needle search tool now also shows snippets where available, providing a nice context for the matched content

Remember that FTS searches apply to any property that’s specified by the ontology as tracker:fulltextIndexed, you can run this to find out:

tracker-sparql -q "select ?prop rdfs:label(?prop) tracker:weight(?prop) where { ?prop tracker:fulltextIndexed true }"


There’s also the possibility you had no idea what I’m talking about , If desktop semantic search still tickles your curiosity I’ll point you to the fine gathered documentation about Tracker.

This work was kindly sponsored by Lanedo.

Performance of a C++11 Signal System

First, a quick intro for for the uninitiated, signals in this context are structures that maintain a lists of callback functions with arbitrary arguments and assorted reentrant machinery to modify the callback lists and calling the callbacks. These allow customization of object behavior in response to signal emissions by the object (i.e. notifying the callbacks by means of invocations).

Over the years, I have rewritten each of GtkSignal, GSignal and Rapicorn::Signal at least once, but most of that is long a time ago, some more than a decade. With the advent of lambdas, template argument lists and std::function in C++11, it became time for me to dive into rewriting a signal system once again.

So for the task at hand, which is mainly to update the Rapicorn signal system to something that fits in nicely with C++11, I’ve settled on the most common signal system requirements:

• Signals need to support arbitrary argument lists.
• Signals need to provide single-threaded reentrancy, i.e. it must be possible to connect and disconnect signal handlers and re-emit a signal while it is being emitted in the same thread. This one is absolutely crucial for any kind of callback list invocation that’s meant to be remotely reliable.
• Signals should support non-void return values (of little importance in Rapicorn but widely used elsewhere).
• Signals can have return values, so they should support collectors (i.e. GSignal accumulators or boost::signal combiners) that control which handlers are called and what is returned from the emission.
• Signals should have only moderate memory impact on class instances, because at runtime many instances that support signal emissions will actually have 0 handlers connected.

For me, the result is pretty impressive. With C++11 a simple signal system that fullfils all of the above requirements can be implemented in less than 300 lines in a few hours, without the need to resort to any preprocessor magic, scripted code generation or libffi.

I say “simple”, because over the years I’ve come to realize that many of the bells and whistles as implemented in GSignal or boost::signal2 don’t matter much in my practical day to day programming, such as the abilities to block specific signal handlers, automated tracking of signal handler argument lifetimes, emissions details, restarts, cancellations, cross-thread emissions, etc.

Beyond the simplicity that C++11 allows, it’s of course the performance that is most interesting. The old Rapicorn signal system (C++03) comes with its own set of callback wrappers named “slot” which support between 0 and 16 arguments, this is essentially mimicking std::function. The new C++11 std::function implementation in contrast is opaque to me, and supports an unlimited number of arguments, so I was especially curious to see the performance of a signal system based on it.

I wrote a simple benchmark that just measures the times for a large number of signal emissions with negligible time spent in the actual handler.

I.e. the signal handler just does a simple uint64_t addition and returns. While the scope of this benchmark is clearly very limited, it serves quite well to give an impression of the overhead associated with the emission of a signal system, which is the most common performance relevant aspect in practical use.

Without further ado, here are the results of the time spent per emission (less is better) and memory overhead for an unconnected signal (less is better):

Signal System	Emit() in nanoseconds	Static Overhead	Dynamic Overhead
GLib GSignal	341.156931ns	0	0
Rapicorn::Signal, old	178.595930ns	64	0
boost::signal2	92.143549ns	24	400 (=265+7+8*16)
boost::signal	62.679386ns	40	392 (=296+6*16)
Simple::Signal, C++11	8.599794ns	8	0
Plain Callback	1.878826ns	-	-

 

Here, “Plain Callback” indicates the time spent on the actual workload, i.e. without any signal system overhead, all measured on an Intel Core i7 at 2.8GHz. Considering the workload, the performance of the C++11 Signals is probably close to ideal, I’m more than happy with its performance. I’m also severely impressed with the speed that std::function allows for, I was originally expecting it to be at least a magnitude larger.

The memory overhead gives accounts on a 64bit platform for a signal with 0 connections after its constructor has been called. The “static overhead” is what’s usually embedded in a C++ instance, the “dynamic overhead” is what the embedded signal allocates with operator new in its constructor (the size calculations correspond to effective heap usage, including malloc boundary marks).

The reason GLib’s GSignal has 0 static and 0 dynamic overhead is that it keeps track of signals and handlers in a hash table and sorted arrays, which only consume memory per (instance, signal, handler) triplet, i.e. instances without any signal handlers really have 0 overall memory impact.

Summary:

• If you need inbuilt thread safety plus other bells and can spare lots of memory per signal, boost::signal2 is the best choice.
• For tight scenarios without any spare byte per instance, GSignal will treat your memory best.
•  If you just need raw emission speed and can spare the extra whistles, the C++11 single-file simplesignal.cc excels.

For the interested, the brief C++11 signal system implementation can be found here: simplesignal.cc
The API docs for the version that went into Rapicorn are available here: aidasignal.hh

PS: In retrospect I need to add, this day and age, the better trade-off for Glib could be one or two pointers consumed per instance and signal, if those allowed emission optimizations by a factor of 3 to 5. However, given its complexity and number of wrapping layers involved, this might be hard to accomplish.

TL;DR: Show location information retrieved directly from ModemManager in a gnome-shell indicator.

I’ve been wanting to try to write a gnome-shell extension from some time, and I ended up finding enough free time during the past weeks. So I ended up writing a gnome-shell extension which would show information about your current location as provided by your modem. Note that this information is not coming from GeoClue; the new ModemManager1 interface is not integrated yet there.

ModemManager exposes these types of information in the Location interface:

• 3GPP location, which is given as a set of 4 parameters, and specifies which is the current network cell which is giving mobile coverage to the modem:
  • MCC/MNC: Identifies the ‘mobile country code’ and the ‘mobile network code’; i.e. country and operator.
  • LAC and Cell ID: Identifies the ‘location area code’ and ‘cell id’ pair.

  Once all these items are known, a query to an open database of cell IDs can give us the exact location of the tower giving us coverage, and therefore a good approximation of where we are located.

• GPS location, given if the modem has a built-in GPS. Currently Option/HSO modems are supported, as well as QMI-powered modems with the ‘PDS’ service implemented.
• CDMA Base Station location, given as Latitude/Longitude coordinates, and available in QMI-powered modems prepared for CDMA networks. As with the 3GPP one, this location information specifies where the station giving coverage to your modem is located, which is a good approximation of where you are located.

Beware! You’ll need ModemManager >= 0.7 (i.e. git master) in order for this extension to work.

You can download and install the extension in the following link:
https://extensions.gnome.org/extension/569/location-indicator

The sources are kept in a git repository in gitorious.org:
https://gitorious.org/aleksander/gnome-shell-location-indicator

If you find the extension itself mostly useless, you may still want to take a look at the source code to see yet another example of how to talk to DBus interfaces from within gnome-shell, including the use of the new standard ObjectManager interface with a Gio.DBusObjectManagerClient.

Filed under: Development, FreeDesktop Planet, GNOME Planet, GNU Planet, Lanedo Planet, Planets Tagged: 3gpp, cdma, extension, gnome, gnome-shell, gps, location, ModemManager, QMI

About

Due to popular request, I’m putting up a polished version of the backup script that we’ve been using over the years at Lanedo to backup our systems remotely. This script uses a special feature of rsync(1) v2.6.4 for the creation of backups which share storage space with previous backups by hard-linking files.
The various options needed for rsync and ssh to minimize transfer bandwidth over the Internet, time-stamping for the backups and handling of several rsync oddities warranted encapsulation of the logic into a dedicated script.

Resources

The GitHub release tag is here: backups-0.0.1
Script URL for direct downloads: sayebackup.sh

Example

This example shows creation of two consecutive backups and displays the sizes.

$ sayebackup.sh -i ~/.ssh/id_examplecom user@example.com:mydir # create backup as bak-.../mydir
$ sayebackup.sh -i ~/.ssh/id_examplecom user@example.com:mydir # create second bak-2012...-snap/
$ ls -l         # show all the backups that have been created
drwxrwxr-x 3 user group 4096 Dez  1 03:16 bak-2012-12-01-03:16:50-snap
drwxrwxr-x 3 user group 4096 Dez  1 03:17 bak-2012-12-01-03:17:12-snap
lrwxrwxrwx 1 user group   28 Dez  1 03:17 bak-current -> bak-2012-12-01-03:17:12-snap
$ du -sh bak-*  # the second backup is smaller due to hard links
4.1M    bak-2012-12-01-03:16:50-snap
128K    bak-2012-12-01-03:17:12-snap
4.0K    bak-current

Usage

Usage: sayebackup.sh [options] sources...
OPTIONS:
  --inc         make reverse incremental backup
  --dry         run and show rsync with --dry-run option
  --help        print usage summary
  -C <dir>      backup directory (default: '.')
  -E <exclfile> file with rsync exclude list
  -l <account>  ssh user name to use (see ssh(1) -l)
  -i <identity> ssh identity key file to use (see ssh(1) -i)
  -P <sshport>  ssh port to use on the remote system
  -L <linkdest> hardlink dest files from <linkdest>/
  -o <prefix>   output directory name (default: 'bak')
  -q, --quiet   suppress progress information
  -c            perform checksum based file content comparisons
  --one-file-system
  -x            disable crossing of filesystem boundaries
  --version     script and rsync versions
DESCRIPTION:
  This script creates full or reverse incremental backups using the
  rsync(1) command. Backup directory names contain the date and time
  of each backup run to allow sorting and selective pruning.
  At the end of each successful backup run, a symlink '*-current' is
  updated to always point at the latest backup. To reduce remote file
  transfers, the '-L' option can be used (possibly multiple times) to
  specify existing local file trees from which files will be
  hard-linked into the backup.
 Full Backups:
  Upon each invocation, a new backup directory is created that contains
  all files of the source system. Hard links are created to files of
  previous backups where possible, so extra storage space is only required
  for contents that changed between backups.
 Incremental Backups:
  In incremental mode, the most recent backup is always a full backup,
  while the previous full backup is degraded to a reverse incremental
  backup, which only contains differences between the current and the
  last backup.
 RSYNC_BINARY Environment variable used to override the rsync binary path.

See Also

Testbit Tools – Version 11.09 Release

For a while now, I’ve been maintaining my todo lists as backlogs in a Mediawiki repository. I’m regularly deriving sprints from these backlogs for my current task lists. This means identifying important or urgent items that can be addressed next, for really huge backlogs this can be quite tedious.

A SpecialPage extension that I’ve recently implemented now helps me through the process. Using it, I’m automatically getting a filtered list of all “IMPORTANT:”, “URGENT:” or otherwise classified list items. The special page can be used per-se or via template inclusion from another wiki page. The extension page at mediawiki.org has more details.

The Mediawiki extension page is here: http://www.mediawiki.org/wiki/Extension:ListItemFilter

The GitHub page for downloads is here: https://github.com/tim-janik/ListItemFilter

It was something like September 2011, shortly after the last Desktop Summit in Berlin, when I started the work improving ModemManager with the new DBus API, GDBus-based DBus support, port-type-agnostic implementations, dynamic DBus interfaces, built-in org.freedesktop.DBus.ObjectManager interface support, built-in org.freedesktop.DBus.Properties.PropertiesChanged signal support, the new libmm-glib client library, the new mmcli command line tool… Took me around half a year to port most of the core stuff, and some more months to port all the plugins we had. And then, suddenly, git master wasn’t that unfinished thing any more, and we were even implementing lots of new features like GPS support in the Location interface, improved SMS Messaging capabilities, ability to switch Firmware on supported modems, and last but definitely not least, support for new QMI-enabled modems through the just officially released libqmi…

And when I thought it was all done already, I woke up from my dream and realized that not even me was really using the new ModemManager as it wasn’t integrated in the desktop… not even in NetworkManager. So there I was again, with a whole new set of things to fix…

NetworkManager integration

There is already a set of patches available to include integration of the new ModemManager interface in NetworkManager; pending for review in the NM mailing list. This is the second iteration already, after having some of the patches from the first iteration already merged in git master.

This integration is probably the most complex one in the list, as it really has to deal with quite different DBus interfaces, but the overall look of it seems to be quite good from my point of view. With the patches on, NetworkManager exposes a new –with-modem-manager-1 configure switch which defaults to ‘auto’ (compile the MM1 support if libmm-glib found). Note that both the old and new ModemManager would be supported in this case, to easier fallback to the old version if the user experiences problems with the new ModemManager.

All in all, the required changes to NetworkManager are quite well defined, implemented mainly as a new ‘NMModemBroadband’ object and some additional bits to monitor added/removed modems through the org.freedesktop.DBus.ObjectManager interface.

gnome-shell integration

A proper integration in my desktop of choice (gnome-shell based GNOME 3) required to integrate the new interface in the Network indicator of the shell. As in every good day, I first had to deal with several issues I found, but I ended up submitting a patch that properly talks to the new ModemManager1 interface when needed. As with NetworkManager, this patch would support both the old and the new interfaces at the same time.

Not that this was a big change, anyway. The Network indicator only uses the ModemManager interface to grab Operator Name, MCCMNC and/or SID, and signal strength. Something that could even be included directly in the NMDevice exposed by NetworkManager actually…

Right now the only big issue pending here is to properly unlock the modem when it gets enabled, something that I still need to check how to do it or where to do it.

Truth be told, hacking the shell has ended up being quite nice, even with my total lack of knowledge in JavaScript. My only suggestion here would be: get a new enough distribution with a recent enough GNOME3 (e.g. the unreleased Fedora 18) and jhbuild buildone from there. I think I’ve never seen a clean jhbuild from scratch succeed… maybe it’s just bad luck.

And then came the applet…

network-manager-applet integration

Why the hell would Bluetooth DUN not work with my implementation, I was thinking, until I found that even if GNOME3 doesn’t rely on nm-applet, it still uses some of its functionality through libnm-gtk/libnma, like the code to decide which kind of modem we’re dealing with and launch the appropriate mobile connection wizard. Ended up also submitting patches to include the missing functionality in network-manager-applet.

For those not using GNOME3 and gnome-shell; note that I didn’t implement the support for the new ModemManager1 interface in the nm-applet itself; I just did the bluetooth-guess-my-modem-type bits required in libnma. If I feel in the mood I may even try to implement the proper support in the applet, but I wouldn’t mind some help with this… patches welcome! Same goes for other desktops relying on NetworkManager (KDE, xfce…); I wouldn’t mind to update those myself as well, but I truly don’t have that much free time.

gnome-control-center integration

Ok, so gnome-shell integration is more or less ready now; we should be finished, right? Well, not just yet. gnome-control-center also talks to ModemManager, in this case to get Operator Name and Equipment Identifier, which btw were not getting properly loaded and updated. Once that fixed, I finally got a new patch to have the control center talk to the new interface.

Fedora 18 packages

I bet you won’t go one by one to all the patches I linked before and apply them in your custom compiled NetworkManager, gnome-shell, network-manager-applet and gnome-control-center… but for all those brave Fedora 18 users, you can try with the 64-bit packages that I built for me, all available here:

• http://www.lanedo.com/~aleksander/fedora-packages/20121112/

If you want to rebuild these packages yourself, you’ll find the source tarballs here and the packaging repositories here. The packaging is really awful – I suck at it – so you’ll probably need to install the RPMs with –force Note that the git repo for packaging has several git submodules, one for each item packaged, so remember to “git submodule init” and “git submodule update“.

We done yet?

Are these all the patches needed to have the best ModemManager experience in GNOME3? No! The list of pending tasks for this purpose grows a bit every day…

• Common code to parse the mobile broadband providers database, exposed in libnm-gtk and used by the wizard, the control center and the shell
• Link mobile connection profiles to a specific SIM/modem
• Ensure modems are PIN-unlocked when they get enabled
• Rework the logic behind the Network indicator in gnome-shell, trying to make it easier to connect to a default connection profile. The same logic would then be need to be applied to the control center…
• Proper IPv6 support in mobile broadband connections
• Connection profiles with multiple APN/bearers
• SMS messaging
• Location information integration
• An independent UI application to manage advanced settings in ModemManager?
• …

Help!

Thanks

Special thanks go to my employer Lanedo GmbH, which sponsors quite a lot of the GNOME integration work; as well as to my girlfriend’s employer, which sends her 400km away from home from Monday to Thusday. Without them this job would have been impossible!


Filed under: Development, FreeDesktop Planet, GNU Planet, Lanedo Planet, Planets Tagged: fedora, gnome, gnome-control-center, gnome-shell, libqmi, ModemManager, NetworkManager, rpm

So, I’ve been having fun traversing the LibreOffice .docx and .rtf import filters while trying to implement Ink annotations in LibreOffice Writer. As it turns out, I don’t strictly agree with the ISO press release that the Office Open XML file format is “intended to be implemented by multiple applications on multiple platforms”. However, despite having spent way too much time wading through a ~24k line XML file defining the document model in the importer, I’ve been very grateful that the format is XML-based and therefore human readable.

I’ve included some useful resources at the end for any intrepid programmers who wish to help with tackling the importer beast.

DOCX import

Ink allows you to annotate using a stylus on a tablet PC using Microsoft Word so that you can doodle over your documents as you see fit. Technical details ahead, so feel free to skip to the results.

Ink strokes are saved in docx documents as bezier curves expressed through VML paths (these are pretty similar to SVG paths, with commands and co-ordinates). I had quite a bit of fun hacking a parser together – here’s the patch, with a few tweaks, it could be generally useful. It produces a list of all the sub-paths in a path, each subpath consisting of a list of co-ordinates and co-ordinate flags indicating normal or control points.

Word’s storage of Ink annotations does highlight some of the problems with implementing Word-compatible OOXML. They’re represented something like this:

<v:shape path="[VML path]" ...>
...
<o:ink i="[base64 binary data]" annotation="t"/>
</v:shape>

Now, [VML path] is really the important part as it contains the Ink shape geometry. But the [base64 binary data]? What’s stored in there is anybody’s guess – I’ve certainly not found any documentation on its contents. Anyone who has a tablet version of Word should feel free to take a crack at reverse engineering it ;)

It turns out that paths for Ink annotations consist of bezier curves. Beziers weren’t supported in the importer, so the path attribute, as well as any <v:curve> elements (which use control1, control2, to and from attributes), got ignored. So I added the support by getting the path and control1/control2 attributes and passing off the parsed result to LibreOffice using the UNO API.

RTF import

Word allows you to export a document with Ink to RTF. Most of the code for importing the RTF equivalent was already there, it just needed some adapting. I found something interesting that I haven’t seen documented elsewhere, furthering Miklos Vajna’s work (see README) on understanding the RTF spec. The geometry of the Ink shapes is described using the pVerticies [sic] and pSegmentInfo keywords. The pSegmentInfo section is a list of commands indicating what the points listed in pVerticies mean (move to, curve, end sub-path and so on).

Segment indicator	Description	Vertices associated
0×0001	Line to point	1 (x, y)
0×2001	Bezier curve with two control points and end point	3 (cx1, cy1, cx2, cy2, x, y)
0×4000	Move to point	1 (x, y)
0×6001	Close path	0
0×8000	End path	0

The plot thickens…

So, when importing a Word-generated .rtf with Ink annotations, why was I seeing segment indicators like 0x200A? Apparently, the low order bytes of certain segment indicators indicate the number of point sets to apply to – for example, if there were four curves in a row with three points each in pVerticies, it can be specified by using the low order bytes of the segment indicator, resulting in 0×2004 (encompassing 12 points in total). This may also apply to other relevant line segment types, but this is as of yet untested. You can easily extract the number of segments indicated using basic bitwise operators:

unsigned int segment = 0x200A; // Example segment indicator
unsigned int points = segment & 0x00FF; // Assuming two lowest order bytes are used for point count
segment &= 0xFF00; // Discard point count; just leave segment indicator


Woo ink in LibreOffice!!

LibreOffice now correctly displays not only Ink, but (in theory) any curves and shapes with paths when importing from .docx or .rtf. A minor bug with RTF image wrapping which caused the shape to be inline with the text instead of over it was also fixed (the property was just being ignored), so better imports all round!

Next step – correct export of bezier shapes to docx and rtf (no, I’m still not sure whether that blob of binary in the o:ink element is of any importance whatsoever, but this should be one way to find out).

Resources

v:shape schema information – this website is great for making sense of the OOXML standard, particularly if used alongside this:
ISO IEC 29500 – ISO standard document for OOXML (warning, big pdf in a zip).
RTF spec – only somewhat useful
UNO API reference – useful if used with the search function
writerfilter and oox – LibreOffice modules of interest for importing OOXML documents (cgit links for browsing the source/READMEs)

Lots of new SMS-related features have landed in ModemManager this week, and as I have as much memory as a fish used to have, I need to write down all the commands I’ve used to test all this; hence this post.

Note that all the stuff I talk about here applies to both AT and QMI based modems.

 

Multiple storages

The current stable 0.5/0.6 releases will only use the core/plugin defined ‘default’ storages for reading and sending short messages. This means that if you happen to have stored SMS in another non-default storage ModemManager won’t show them to you.

In the new 0.7 codebase, ModemManager reads the messages from all the available storages. It is able to read not only DELIVER-type PDUs from received SMS, but also SUBMIT-type PDUs from user-created SMS which got stored. The “Sms.Store()” method in the interface now allows (optionally) specifying in which storage the SMS should be kept.

The following commands allow to create a new SMS and store it in the “sm” (SIM) storage:

$> sudo mmcli -m 0 \
              --messaging-create-sms="text='Hello world',number='+1234567890'
Successfully created new SMS:
    /org/freedesktop/ModemManager1/SMS/21 (unknown)
 
$> sudo mmcli -s 21 --store-in-storage="sm"
successfully stored the SMS
 
$> sudo mmcli -s 21
SMS '/org/freedesktop/ModemManager1/SMS/21'
  -----------------------------------
  Content    |              number: '+1234567890'
             |                text: 'Hello world'
  -----------------------------------
  Properties |            PDU type: 'submit'
             |               state: 'stored'
             |                smsc: 'unknown'
             |            validity: '0'
             |               class: '0'
             |             storage: 'sm'
             |     delivery report: 'not requested'
             |   message reference: '0'


Note that you can create, store and send SMS objects regardless of what storage you’re using for each, ModemManager will take care of properly synchronizing the access to each of them.

The Messaging interface also defines new properties to list all the supported storages and to specify which is the default storage being used for received messages (and for stored ones when no specific one given in “Sms.Store()“):


$> sudo mmcli -m 0 --messaging-status
/org/freedesktop/ModemManager1/Modem/0
  ----------------------------
  Messaging | supported storages: 'sm, me'
            |    default storage: 'me'


 

Text multipart SMS

We support receiving text multipart messages in ModemManager since some stable releases ago; but support for sending multipart messages was still missing. Not any more! Roberto Majadas of OpenShine started this work some months ago with a patch for MM 0.6, which I took over and ported to git master and the new 0.7 codebase.

Of course, we support both the default GSM 7-bit alphabet, as well as UCS-2:

• GSM 7 encoded multipart messages: If the message is less or equal than 160 septets a singlepart SMS will be used; otherwise the input text will be split into chunks of 153 septets and sent as a multipart message.
• UCS-2 encoded multipart messages: The GSM 7 alphabet and its extensions are not enough to cover all scripts available out there. Luckily, we can send PDUs with UCS-2 encoded text (UCS-2 covers all the basic plane of Unicode, and is equivalent to UTF-16 in that range). When using this encoding, if the input text is less or equal than 70 UCS-2 characters (140 bytes) a singlepart SMS will be used; otherwise the input text will be split into chunks of 67 UCS-2 characters (134 bytes) and sent as a multipart message. Note anyway that this doesn’t mean that we only allow 70-byte long input strings to fit into a singlepart message; as the conversion from the input UTF-8 text to UCS-2 isn’t always increasing the required number of bytes. As an example, we can now send 210 bytes of UTF-8 encoded chinese text into a single SMS with 140-bytes of UCS-2.

Note that from the API user’s point of view, the SMS objects are not considered multipart or singlepart; that is taken as internal information only required for the PDU-based implementation.

Multipart messages can be created by writing a long enough ‘text‘ value passed in the ‘--messaging-create-sms‘ command, as shown before.

 

Raw data multipart SMS

In addition to text short messages we now also properly support raw binary data singlepart and multipart messages (reported as 8bit encoded). This means that both the sender and receiver can agree in any text encoding to be used between them, or even send in one or more SMS whatever raw binary stream you have.

The ‘mmcli’ command line utility was improved with several commands to test this feature. For example, you can now send using just SMS messages whatever file you have in your system, just doing:

$> sudo mmcli -m 0 \
       --messaging-create-sms="number='+1234567890'" \
       --messaging-create-sms-with-data=/path/to/your/file
Successfully created new SMS:
    /org/freedesktop/ModemManager1/SMS/22 (unknown)
 
$> sudo mmcli -s 22 --send
successfully sent the SMS


When the receiver gets all the parts of the message, she can now recover the sent file with another ‘mmcli’ command in her ModemManager setup:

$> sudo mmcli -m 0 \
              --messaging-list-sms
Found 1 SMS messages:
    /org/freedesktop/ModemManager1/SMS/0 (received)
 
$> sudo mmcli -s 0 \
              --create-file-with-data=/path/to/the/output/file


But beware! The file is sent in chunks of 134 bytes, and there is a maximum of 255 parts in a given multipart SMS, so this leaves you with a maximum size of around 33 Kbytes And not only that, if you’re paying e.g. 0.10€ for each SMS sent, you would be paying more than 25€ for those 33 Kbytes. Yes, needless to say, this is therefore the most useless use case ever. But still, works as a proof of concept.

 

Delivery reports

Last and least, you can now also request delivery reports when sending an SMS, by passing the ‘delivery-report-request=yes‘ key/value pair in the ‘--messaging-create-sms‘ command.


$> sudo mmcli -m 0 \
           --messaging-create-sms="text='你好',\
                                   number='+1234567890',\
                                   delivery-report-request=yes"
Successfully created new SMS:
  /org/freedesktop/ModemManager1/SMS/0 (unknown)
 
$> sudo mmcli -s 0 --send
successfully sent the SMS
 
$> sudo mmcli -m 0 --messaging-list-sms
Found 21 SMS messages:
  /org/freedesktop/ModemManager1/SMS/0 (sent)
  /org/freedesktop/ModemManager1/SMS/1 (received)
 
$> sudo mmcli -s 1
SMS '/org/freedesktop/ModemManager1/SMS/1'
  -----------------------------------
  Content    |              number: '+1234567890'
  -----------------------------------
  Properties |            PDU type: 'status-report'
             |               state: 'received'
             |                smsc: '+9876543210'
             |            validity: '0'
             |               class: '0'
             |             storage: 'unknown'
             |   message reference: '137'
             |           timestamp: '120914120619+02'
             |      delivery state: 'temporary-error-sc-specific-reason' (0x30)
             | discharge timestamp: '120914120619+02'


You can then match the delivery report with the message sent by using the ‘message reference‘ value, which should be the same in both. This also applies to multipart messages, where the ‘message reference’ of our SMS object is the one reported when sending the last part, and therefore we only request the delivery report in the last part sent.

Enjoy!

Filed under: Development, FreeDesktop Planet, GNU Planet, Lanedo Planet, Planets Tagged: mmcli, ModemManager, NetworkManager, QMI, SMS

When concluding the discussion on reference counting rules for nodes in an earlier blog post, we noted that these rules put limits on what kind of visible functions will function properly. The filter model must follow changes in its child model to keep the internal data structure up to date. As we have seen, changes can only be expected to be received for levels in which the filter model has a reference on at least one node. By default any references obtained on filter model nodes are passed on to the child model. So, the filter model will receive signals for all nodes that are visible in the filter model’s observer.

This is not enough to support an often used use case. If we, for example, want to only show a parent node if one of its children is visible, we need to monitor signals received on the child nodes, also when the parent node is hidden. Once one of the children changes state such that it will become visible, we must make the parent visible as well. Clearly, when the parent is invisible, its children are not referenced by the filter model’s observer.

Because this use case is so often used, we have made GtkTreeModelFilter explicitly support it. This is done by building child levels in the internal data structure of each node that could be visible. When a child level is built, a reference is obtained on the first node in that level. A node could be visible if any node in the level it is contained in (so any sibling) has been referenced by the filter model’s observer. Note that we thus look at the reference count of a level here and not at the reference count of individual nodes. We must also build the child level for invisible nodes contained in a visible level. Such child levels are monitored for changes and for every signal received a check is done whether this changed the visibility state of the parent node.

This explicit support is limited to dependencies from a node on its child for the node’s visibility state. If you want a node’s visibility to depend on the children of the node’s immediate children, you are on your own. In such cases you can either rely on GtkTreeStore, which always emits signals for all nodes because it does not implement reference counting, or obtain references on the additional child levels yourself.

To wrap up, we can in general say that visible functions should only use data or properties from the node for which the visibility state must be determined, its siblings or its parents. Because of the explicit support for this, it may also use data or properties from its children. If any other data is used, the filter model may not work reliably.

 

With this post, we have come to the end of the blog post series on peculiarities learned when fixing up GtkTreeModelSort and GtkTreeModelFilter last Summer. If you have any questions about these posts, do feel free to
contact me.

GtkTreeModelSort and GtkTreeModelFilter basically proxy the shape of their child model, including modifications. Internally, the models proxy incoming requests from their observers to requests on the child model. A mapping is needed from nodes exposed by the sort and filter models to nodes in their child models. Since the models can “perform” modifications, the mapping must be able to deal with these modifications.

The mapping is internally implemented as a tree of nodes that are visible in the observer (i.e. the observer took references on these nodes). Each level is an array of nodes. When we hand out an iterator to the observer, the user data of the iterator is set to a pointer to the node in the array. When the iterator is used, we can very quickly access the node in the internal data structure. Using data in this node (for example index into the level in the child model) we can construct an iterator relative to the child model to forward the operation. Sometimes we can even cache these iterators. A node can have a “children” pointer set to point at a child level.

Using arrays works well — we have been using GArray for a long time — unless the internal data structure is subject to frequent changes. This either happens due to a lot of activity in the child model, or due to filtering activity. When adding and removing nodes in the GArray, memory has to be moved. When the array is large and this happens often, this becomes very costly.

Xavier Claessens noticed this and wrote a patch to replace the use of GArray with GSequence. This significantly sped up operation of the filter model in certain cases. We have committed this change as part of the “treemodel-fix” branch last year, but only after we got a lot more of the sort and filter model under unit test. This way, we could ensure that no regressions were introduced when migrating from GArray to GSequence.

A couple of rules must be played by when using reference counting of GtkTreeModel nodes. These rules very much make sense when you reason about them. Also, the rules are paramount to implementing something like GtkTreeModelFilter, as we will see later. Let us first discuss the rules:

1. Never take a reference on a node, without owning a reference on its parent. This means that all parent nodes of a referenced node must be referenced as well. Looking at this from a GtkTreeView perspective, this rule will never be violated. GtkTreeView will always take a reference on every node that is present in GtkTreeView’s RB tree, which contains all children of expanded parents. Naturally, GtkTreeView cannot expand a parent which is not visible itself, so parent nodes are always referenced.

   When you are implementing your own GtkTreeModel, you are free to reference whatever you want. In this case, it becomes important to adhere to the rules. When GtkTreeModelSort or GtkTreeModelFilter is set as the child model of your custom model, things quickly go wrong if you do not play by the rules. The sort and filter models aggressively empty their internal data structures of nodes that are no longer needed. This process is based on the current reference counts of the nodes. So, when you reference child nodes without holding a reference on their parents, a parent might eventually end up at a reference count of zero while its children are still visible. A parent level might wrongly be pruned from the cache, corrupting the data structures. (Of course, the sort and filter models could enforce the rule for you by referencing all parents when a node is reference, but this significantly hampers performance because many more virtual ref/unref calls are done than necessary).

2. Outstanding references on a deleted node are not released. So, once you receive a row-deleted signal for a node in your model, do not expect any unref call for that node to follow. You can implicitly erase all references.

   If you are not the model implementor, but instead have a couple of references on nodes in this model, then do not release references of a node once you have received the signal that that node has been deleted. (This actually implies you will have to listen for row-deleted signals if you have references on nodes, or you use a GtkTreeRowReference which will be set to NULL once the node has been deleted).

   It is clear why this rule is there: by the time you receive a row-deleted signal for a node, the node no longer exists in the model so you cannot get an iter to this node to perform the unref call.

3. Models are not obligated to emit a signal on rows of which none of its siblings are referenced. To phrase this differently, signals are only required for levels in which nodes are referenced. For the root level however, signals must be emitted at all times (however the root level is always referenced when any view is attached).

   What is this rule trying to say at all? Imagine you have a ref-counted model, some node in the root level has a couple of child nodes and only the nodes in the root level are referenced. This means that none of the nodes is currently expanded. The rule says that, as a result, any signals for the child nodes (e.g. row-changed if a child nodes changes) do not have to be emitted.

   Makes sense — what view is interested in signals of nodes that are not displayed in the view anyway? In general, you are indeed not interested. But the for GtkTreeModelFilter this rule is very important in order to properly handle a sensible use case. Imagine that you have a filter on child nodes. You only want to show an expander arrow next to the parent node if any of the child nodes is visible. To handle the expander arrow correctly, we must receive events for the child nodes, also when these nodes are not referenced. GtkTreeModelFilter handles this by explicitly obtaining a reference on one of the child nodes if a parent is referenced. Due to this rule, a reference on a single node is sufficient to start receiving signals for this level.

In summary, we can say that good understanding of these rules, which seem so obvious, is crucial when implementing a model like GtkTreeModelFilter. The rules are used to determine when it is required to emit signals and when levels in the internal data structures can be cleaned up. I can go into more detail on what is involved in GtkTreeModelFilter, but that is probably not of that much interest. What is for example required is to for each node maintain two reference counts; it’s actual, real, reference count and it’s external reference count. The external reference count maintains the reference count based on the ref and unref calls received from the model’s observer. This determines a node’s visibility, so should not be mixed up with any reference on the node taken internally!

As we have seen, these rules also put limits on what can be done with a visible function in GtkTreeModelFilter. More on that in a later blog post.

Many people are confused by what exactly the reference counting methods ref_node() and unref_node() in the GtkTreeModel interface are intended for. These methods are a way for views (or more generically, model observers) to let models know when nodes are being displayed. For example, GtkTreeview will obtain a reference on a node when the node is visible and release the reference when the node becomes invisible. And this is immediately the main source of the confusion: the confusion typically centers around the exact definition of node visibility.

There are two definitions of visibility in use:

1. A node is visible when it is visible on the screen, that is, the user can currently see it. This is the most obvious definition of this term. For a node to be visible according to this definition, the node’s parent must be in expanded state, secondly, the coordinates of the node must be within the coordinates that are currently visible in the tree view’s scrollable viewport.
2. A node is visible when its parent node is in the expanded state (or the node is in the root level in which case you can say that it is always visible because its implicit parent is always in expanded state). Do note that when a node is visible according to this definition, it might not be immediately visible to the user! This definition does not say anything about the current position of the scrollbar.

For the reference counting mechanism of GtkTreeModel, the second definition is in use. So, what is the point exactly in knowing whether a node’s parent is expanded? The design rationale of the reference counting mechanism is to be able to delay loading content of child nodes until it is really necessary. A useful use case is a file system model. You do not want to load in an entire file system tree when the model is being instantiated. Instead, only the root level is loaded. All nodes in the root level will be referenced when the model is attached to a view. In response to this, one could decide to load ahead the child nodes of the root level nodes. Once a root level node is expanded, the children of this node get referenced, and so on. Similarly, when nodes are collapsed, nodes are unreferenced. This information can be used to possibly prune nodes from the tree which are no longer necessary to keep in memory.

Rather unfortunate is that usually you want to avoid loading child nodes if the parent node has not yet been expanded. Especially when you are for example operating on a remote file system. A preferred approach is often to delay loading the nodes until the parent is expanded, displaying a “Loading …” node in the level until the real child nodes are retrieved. The reference counting mechanism does not really aid you to implement that, except that you can re-use it for cache pruning.

The reference counting scheme is not of much use for lists either. Generally, for lists, developers are mostly interested in the first definition of visible. When dealing with lists with large numbers of nodes, you want to avoid loading all data from the data source if this is expensive. Alas, because all nodes in the root level get referenced as soon as the model is attached to a view and there is no relation between the reference count and whether a node is visible on the screen, other methods have to be used to implement this. The fact that models are separated from views and that there is no mechanism to relay information about which nodes are visible on the screen to a model, makes it hard to implement this. What is really needed is a proper “lazy loading model interface”, such that views and models can exchange information on this.

The scheme does work very well for GtkTreeModelSort and GtkTreeModelFilter. These models actually must load child nodes as soon their parent is referenced. GtkTreeModelFilter must process the child nodes (i.e. performing the filter function on these nodes) in order to determine whether the child nodes are visible at all and thus whether the parent node actually has children. Recall that as soon as a node is visible, it is queried whether it has any children such that an expander can be drawn next to the node.

I have a confession to make. Years ago, I told people who wrote GtkTreeModels that implemented reference counting to emit GtkTreeModel:row-deleted before actually removing the node from the model’s internal data structures, contrary to after like the documentation suggested. GtkTreeModelFilter and GtkTreeModelSort have worked like this for years. It turns out I have been wrong, and I now finally understand why — I failed to correlate two commits which fixed related bugs back in 2002.

Back in February 2002, emitting row-deleted after deleting the node from the internal data structure was indeed a problem and caused warnings/errors to be spewed to the console. I (wrongly) reasoned that the nodes had to be available during the row-deleted phase to receive the unref calls, and added a comment:

  /* we _need_ to emit ::row_deleted before we start unreffing the node
   * itself. This is because of the row refs, which start unreffing nodes
   * when we emit ::row_deleted
   */

I continued to refer to this comment even in 2007.

During that e-mail thread, I did notice a single line in gtktreemodel.c mentioning that “Please note that nodes that are deleted are not unreffed.”. But clearly, the contrary was happening in February 2002? Deleted nodes were being unreffed.

It turns out that these faulty unrefs were caused by another bug which was fixed in April 2002! Unfortunately, I failed to correlate these commits and continued to be wrongly convinced that emitting row-deleted after really removing the node could be troublesome.

The good thing is that this is now documented together with other rules to take into account when dealing with reference counting (blog post on this will follow) and it is now unit tested, so it cannot happen again.

Table of Contents

I’m working on allowing a Windows Installer (.msi) for LibreOffice to be built when cross compiling under Linux. So far, it has been a broad spanning project and has covered:

• Windows, MSI and Cabinet APIs (C, SQL, Wine, winegcc)
• LibreOffice build system (Perl, autotools)

Project status as of posting:

• Developing on openSUSE 12.1 (x86_64) to target Windows (i686).
• .msi files can be created and taken apart with the cross MSI tools (cgit) msidb and msiinfo.
• Cabinet files can be extracted but not created. However, parsing Diamond Directive file (.ddf) format is supported through makecab (this is required when the LO build system creates a cabinet).
• Remaining: hook up MSI transforms and patches (msitran and msimsp); fit the tools into the build system; clean up and maintenance.

The MSDN documentation for the win32 native tools has been linked to where appropriate.

1.1 Cross compiling LibreOffice

Luckily, LibreOffice cross compile support is already very good. README.cross in the LibreOffice root directory has far more information. Assuming you have checked out LibreOffice and have all the MinGW dependencies, cross compiling can be as simple as changing <lo_root>/autogen.lastrun to read:

CC=ccache i686-w64-mingw32-gcc
CXX=ccache i686-w64-mingw32-g++
CC_FOR_BUILD=ccache gcc
CXX_FOR_BUILD=ccache g++
–with-distro=LibreOfficeMinGW

This references <lo_root>/distro-configs/LibreOfficeMinGW.conf. This folder
contains various configurations for compiling under different circumstances. I
also found it helpful to add this line to LibreOfficeMinGW.conf to make life
simpler:

–without-java

1.2 Building the installer

The installer build logic can be found in <lo_root>/solenv/bin/modules/installer/windows. It makes use of several Microsoft utilities to eventually output an MSI file. Some of these utilities are already distributed by Wine.
Provided by Wine:
expand.exe – Used to unpack cabinet files.
cscript.exe – Command line script host.
Also expected:
msidb.exe – Manipulates installer database tables and streams.
msiinfo.exe – Manipulates installer meta data (summary information).
makecab.exe – Compresses files into cabinets.
msimsp.exe – Creates patch packages.
msitran.exe – Generates and applies database transforms.

Wine already exposes most of the required functionality via the API exposed by msi.dll (MSDN, Wine) and cabinet.dll (MSDN, Wine). My work has been focussed on writing command line utilities that support the interface expected by the LibreOffice build scripts.

• solenv/bin/make_installer.pl is a very large Perl script that connects up the Perl modules which build the installer. The .pm files relevant to cross MSI building are listed below.
• solenv/bin/modules/installer/control.pm performs “nativeness” logic such as checking if the environment is Cygwin and whether the required utilities are in the system path.
• solenv/bin/modules/installer/windows/admin.pm (expand.exe*, msidb.exe, msiinfo.exe)
• solenv/bin/modules/installer/windows/mergemodule.pm (expand.exe*, msidb.exe)
• solenv/bin/modules/installer/windows/msiglobal.pm (msidb.exe, msiinfo.exe, cscript.exe*, msitran.exe**, makecab.exe**)
• solenv/bin/modules/installer/windows/msp.pm (msidb.exe, msimsp.exe**)
• solenv/bin/modules/installer/windows/update.pm (msidb.exe)

* Distributed by Wine
** In progress

1.3 Cross MSI tool development

The code for these tools can be found in the the feature/crossmsi branch of libreoffice. It currently resides in setup_native/source/win32/wintools in the tree.

To test the tools individually, grab the dev Makefile and make from the tool’s directory. You can then pass the -? or /? command for usage. I would suggest disabling Wine’s debug logs unless you specifically need them:

$ export WINEDEBUG=-all

• msidb (MSDN msidb, LibreOffice msidb, dev Makefile)
  
  Usage: msidb [options] [tables]

  Options:
  -d <path> Fully qualified path to MSI database file
  -f <wdir> Path to the text archive folder
  -c Create or overwrite with new database and import tables
  -i <tables> Import tables from text archive files – use * for all
  -e <tables> Export tables to files archive in directory – use * for all
  -x <stream> Saves stream as <stream>.idb in <wdir>
  -a <file> Adds stream from file to database
  -r <storage> Adds storage to database as substorage

• msiinfo (MSDN msiinfo, LibreOffice msiinfo, dev Makefile)
  
  Usage: msiinfo {database} [[-b]-d] {options} {data}

  Options:
  -c <cp> Specify codepage
  -t <title> Specify title
  -j <subject> Specify subject
  -a <author> Specify author
  -k <keywords> Specify keywords
  -o <comment> Specify comments
  -p <template> Specify template
  -l <author> Specify last author
  -v <revno> Specify revision number
  -s <date> Specify last printed date
  -r <date> Specify creation date
  -q <date> Specify date of last save
  -g <pages> Specify page count
  -w <words> Specify word count
  -h <chars> Specify character count
  -n <appname> Specify application which created the database
  -u <security> Specify security (0: none, 2: read only 3: read only (enforced)

• makecab (MSDN makecab, LibreOffice makecab, dev Makefile)
  
  Usage: makecab [/V[n]] /F directive_file

  Options:
  /F directives – A file with MakeCAB directives.
  /V[n] – Verbosity level (1..3)

 

Like every year, I am driving to Berlin this week to attend LinuxTag 2012 to attend the excellent program. If you want to meet up and chat about projects, technologies, Free Software or other things, send me an email or leave a comment with this post and we will arrange for it.

Once upon a time, like 5 weeks ago, there used to be the longstanding plan to, at some point in the future, port GIMP to GEGL.

We have done a lot of refactoring in GIMP over the last ten years, but its innermost pixel manipulating core was still basically unchanged since GIMP 1.2 days. We didn’t bother to do anything about it, because the long term goal was to do all this stuff with GEGL, when GEGL was ready. Now GEGL has been ready for quite a while, and the GEGL porting got assigned a milestone. Was it 2.10, 3.0, 3.2, I don’t remember. We thought it would take us forever until it’s done, because nobody really had that kind of time.

About 5 weeks ago, I happened to pick up Øyvind Kolås, aka Pippin the Goatkeeper to stay at my place for about a week and do some hacking. After one day, without intending it, we started to do some small GEGL hacking in GIMP, just in order to verify an approach that seemed a good migration strategy for the future porting.

The Problem: All the GimpImage’s pixels are stored in legacy data structures called TileManagers, which are kept by high level objects called GimpDrawables. Each layer, channel, mask in GIMP is a GimpDrawable.

A typical way to do things is:

TileManager *tiles = gimp_drawable_get_tiles (drawable);
PixelRegion region;

pixel_region_init (&region, tiles, x, y, w, h, TRUE);

/* do legacy stuff on the pixel region in order to change pixels */

After the GEGL porting, things would look like that:

GeglBuffer *buffer = gimp_drawable_get_buffer (drawable);

/* do GEGL stuff on the buffer, like running it through a graph in order to change pixels */

Just, how would we get there? Replacing the drawable’s tile manager by a buffer, breaking all of GIMP at the same time while we move on porting things to buffers instead of tile managers? No way!

The Solution: A GeglBuffer’s tiles are stored in a GeglTileBackend, and it’s possible to write tile backends for arbitrary pixel storage, so why not write a tile backend that uses a legacy GIMP TileManager as storage.

After a few hours of hacking, Pippin had the GimpTileBackendTileManager working, and I went ahead replacing some legacy code with GEGL code, using the new backend. And it simply worked!

The next important step was to make GimpDrawable keep around a GeglBuffer on top of its TileManager all the time, and to add gimp_drawable_get_buffer(). And things just kept working, and getting easier and easier the more legacy code got replaced by GEGL code, the more GeglBuffers were being passed around instead of TileManagers and PixelRegions.

What was planned as a one week visit turned into 3 weeks of GEGL porting madness. At the time this article is written, about 90% of the GIMP application’s core are ported to GEGL, and the only thing really missing are GeglOperations for all layer modes.

As a totally unexpected extra bonus, there is now even a GEGL buffer tile backend in libgimp, for plug-ins to use, so also plug-ins can simply say gimp_drawable_get_buffer(drawable_ID), and use all of GEGL to do their stuff, instead of using the legacy pixel region API that also exists on the plug-in side.

GIMP 2.10’s core will be 100% ported to GEGL, and all of the legacy pixel fiddling API for plug-ins is going to be deprecated. Once the core is completely ported, it will be a minor effort to simply “switch on” high bit depths and whatever color models we’d like to see. Oh, and already now, instead of removing indexed mode (as originally planned), we accidentally promoted indexed images to first class citizens that can be painted on, and even color corrected, just like any other image. The code doing so doesn’t even notice because GEGL and Babl transparently handle the pixel conversion magic.

The port lives in the goat-invasion branch in GIT. That branch will become master once GIMP 2.8 is relased, so the first GIMP 2.9 developer release will already contain the port in progress.

If you want to discuss GIMP and GEGL things with us face to face, join us at this year’s Libre Graphics Meeting in Vienna, in two weeks from now, a lot of GIMP people will be there; or simply check out the goat-invasion branch and see the goats yourself.

If you have some Euros to spare, consider donating them to Libre Graphics Meeting, it’s one of the few occasions for GIMP developers, and the people hacking on other projects, to meet in person; and such meetings are always a great boost for development.

During the 3 crazy weeks, quite some work time hours were spent on the port, thanks to my employer Lanedo for sponsoring this via “Labs time”.

Student applications for Google Summer of Code 2012 will be open very soon. After an extremely enjoyable and rewarding experience with the program last year, I feel it’s my duty to student programmers to get the word out. So, here’s why you should apply.

You get paid to work on open source software. I became a long time user, first time contributor early last year. Looking to give something back, I attempted a LibreOffice Easy Hack. In a case of fantastic timing, they announced their involvement in GSoC a week or so later and I got in touch. The end result was a whole new open source library. I had an amazing experience working with LibreOffice but it’s ideal to choose a project that’s personally useful. GSoC doesn’t require that you’re an open source evangelist but if you are, it’s a strong argument for applying.

It’s fantastic experience working on a large project. I feel I learned more during those three months than during my undergraduate degree course. I have to say that I never particularly enjoyed groupwork at university but it’s completely different if you’re working with smart, motivated individuals who’re there either because they want to be or because they’re paid to be (quite often both). As a nice bonus, it’s great work experience and has essentially led me to my dream job. I’m not sure if that’s a typical result, but it certainly wouldn’t hurt to have it on your CV or resume.

You meet some of the smartest, most awesome people (not all of them programmers). I think this is my favourite outcome. I’ve met people from all over the world with an assortment of beliefs, opinions and backgrounds. My experience was that some of the best hackers and coolest people (no, seriously!) hang around open source communities.

Applying isn’t difficult, just choose a participating open source organisation or two and do a little research into the suggested projects before getting in touch with them. Good luck!

So, after a few strives during the last year, the multitouch Xorg patches were posted and merged to master last month, making multitouch available in the upcoming Xorg release. This turns the multitouch GTK+ branch into a suitable candidate for GTK+ 3.4, which obviously deserves a video demoing what’s up there:

Hopefully soon in master, very soon…

We’re currently looking for anyone who has LibreOffice experience and is interested in working on the project. If that sounds like something you would like to do, get in touch with us.

Additionally, if you or anyone you know has experience running an open source business, please get in touch. We’re looking for someone that could facilitate a CEO type position.

I’ve been home a week from the LibreOffice Conference in Paris and from a personal point of view, it was a huge success.

First of all, here are my slides from the short talk I gave about what we achieved with libvisio over the duration of Google Summer of Code. There is still work to be done but once end-user feedback starts coming in, we can sand down any rough edges.

The conference was a lot of fun, particularly the company. I had the pleasure of meeting the rest of the libvisio team, Fridrich Strba and Valek Filippov, who looked out for me the whole time I was there. I’m sure the Paris pickpockets are still cursing their names.

I also have to admit to being a little starstruck at meeting all the fantastic hackers whose work I have made so much use of. The LibreOffice team were a diverse, interesting and kind bunch who put up with my incessant (well-meaning) questions with good grace and gave me plenty to think about on coding, the universe and everything.

It was wonderful to be surrounded by programmers and Linux users without the geekier-than-thou attitude. Despite being younger (and greener) than most and female unlike many (with a few notable exceptions), I chatted away to my fellow hackers without once feeling patronised.

Finally, I’m staying out of the whole political situation – I started coding with LibreOffice for pragmatic reasons (I could get the code easily, Easy Hacks make getting to know the project simpler and LibreOffice was part of GSoC ’11). However, I think the conference really confirmed for me that as important as the code base is, the community that surrounds a project this size is as vital. Without their helpful, inclusive approach, I’d have found contributing to an open source project of that magnitude an insurmountable task.

So here’s to another year!

Given there have been a number of improvements to tracker-needle recently, I thought I would make a video to highlight some of them. A quick summary:

• Searching for “foo” now finds files tagged with “foo”
• Searches are limited to 500 items per category/query (to avoid abusing the GtkTreeView mainly)
• A tag list is now available to show all hits by tags
• Tags can be edited by the context menu per item (planned to be improved later)

Really nice to have tagging supported properly in tracker-needle now.

 

And here’s another muffin from the code cake factory…

About Testbit Tools
The ‘Testbit Tools’ package contains tools proven to be useful during the development of several Testbit and Lanedo projects. The tools are Free Software and can be redistributed under the GNU GPLv3+.

This release features the addition of buglist.py, useful to aid in report and summary generation from your favorite bugzilla.

Downloading Testbit Tools
The Testbit Tools packages are available for download in the testbit-tools folder, the newest release is here: testbit-tools-11.09.0.tar.bz2

Changes in version 11.09.0:

Added buglist, a script to list and download bugs from bug trackers.

Added buildfay, a script with various sub commands to aid release making.

Fixed version information for all tools.

Added support to the Xamarin Bug Tracker to buglist.py.

 

Feedback

If you find this release useful, we highly appreciate your feature requests, bug reports, patches or review comments!

See Also

1. The Bugzilla Utility buglist.py – managing bug lists
2. Wikihtml2man Introduction – using html2wiki

Something I have been meaning to do for a long time, is to update the preferences dialog for Tracker to easily add locations which are special user directories (as per the GUserDirectory locations).

I wanted to do this in such a way that:

• It was really easy to toggle locations as recursive or not
• The file chooser was only necessary for non-standard locations
• Better use of the space was made by integrating the two lists (previously) for single directory and recursive directory indexing
• I could fix a few issues which had been reported when it came to saving using the special symbols (e.g. &DESKTOP for G_USER_DIRECTORY_DESKTOP, etc.) when one or more user directories evaluated to the same location

The result is this (now in master and 0.12.2 when it is released):

Given we (the tracker team) want to try to fit into the GNOME schedule for 3.2, we decided to bring the release of 0.12.0 ahead early. The roadmap is mostly complete anyway.

The official announcement can be seen here.

Thank you to everyone involved!

Recently I also updated the GtkSearchEngineTracker implementation to not use hacky dlopen() calls and to use DBus instead. This avoids us updating the work for each new version of Tracker that comes along too. The patch attached to the bug (658272) should be applied soon (given Matthias was pushing for this sooner rather than later). So, we’re all on track!

Last Thursday I merged the Apple Filing Protocol (AFP) backend for GVfs; so we finally have support for Apple shares too now. It has been written by Carl-Anton Ingmarsson and it was his Summer of Code 2011 project. It is on the master branch and thus will be in the next unstable release. Please test it and report bugs against the "afp backend" component.

Carl-Anton did quite an impressive job - probably best depicted by the diffstat of the merge:

 client/Makefile.am            |    1 
 client/afpuri.c               |  269 ++
 client/gdaemonvfs.c           |    2 
 configure.ac                  |   31 
 daemon/Makefile.am            |   45 
 daemon/afp-browse.mount.in    |    8 
 daemon/afp.mount.in           |    5 
 daemon/gvfsafpconnection.c    | 1651 ++++++++++++++++
 daemon/gvfsafpconnection.h    |  420 ++++
 daemon/gvfsafpserver.c        | 1033 ++++++++++
 daemon/gvfsafpserver.h        |   85 
 daemon/gvfsbackendafp.c       | 4292 +++++++++++++++++++++++++++++++++++++++++-
 daemon/gvfsbackendafp.h       |   23 
 daemon/gvfsbackendafpbrowse.c |  608 +++++
 daemon/gvfsbackendafpbrowse.h |   47 
 daemon/gvfsbackenddnssd.c     |    6 
 daemon/gvfsjobsetattribute.h  |    1 
 17 files changed, 8491 insertions(+), 36 deletions(-)

Recently Adrien Bustany blogged about the Firefox extension for Tracker and has yet to blog about his Thunderbird extension work.

As you would expect, the Firefox extension syncs bookmarks to Tracker (in that direction only for now) and the Thunderbird extension sends email to Tracker to be indexed (even full text content of emails which our Evolution miner doesn’t do because of the system stress it causes). This is really quite superb work from Adrien and tracker-needle already supports bookmarks and emails so it all just works after a make install (into the $prefix where Firefox/Thunderbird are installed). Currently the Thunderbird extension requires version >= 5.0 (works with betas too), and the Firefox extension requires version >= 4.0 (and supports 5.0).

These works have been imported using a pretty cool tool after I felt more comfortable using that to import Adrien’s subtrees into Tracker’s git repository. I did read up on the coolest merge ever from Linus but it felt more like a hack to me to do it that way. Still, I guess Linus knows what he is doing

So now we have both plugins imported with full history into git. The thunderbird branch was merged to master today and the firefox branch will be merged this week hopefully pending Adrien’s review. Great stuff!

So, I have finally made progress that isn’t so ground-breaking that my mentor wants to write about it but is big enough that certain people will stop making fun of my empty blog. So, frob (his wonderfully useful work can be found here), I hope you’re happy.

I’ve been working on shapes, lines and their properties, most recently on fills. Here’s how it’s going so far (Visio document on top, my output below).

Thanks to frob for the image, plus animated gif.

A few technical details for those who care: Visio draws shapes (including rectangles) as individual lines and before they can be filled, so we have to manually detect whether or not it’s a closed polygon. At the moment, we simply take the first point and compare it to the last point and make sure there are no gaps in between. It works for most simple cases but since when are things ever truly simple when reverse engineering?

You may also notice a difference between how gradients 31-34 are drawn in Visio vs my output. There’s no direct equivalent of this type of square gradient that I know of in the SVG or ODG specifications, so we’re approximating it. I have a whole new appreciation of slight imperfections when porting documents to different formats.

In the time it has taken to write this, I’ve already found that some of what I’ve written about will change. This is why I’m a programmer not a blogger ;)

Earlier today I’ve pushed gedit-cossa, a plugin for gedit to help writing CSS for GTK+ themes, it is able to display a number of samples, loaded from GtkBuilder files.

Here’s a video demonstrating how it works:

Cossa is still in pretty early development stages, immediate plans include:

• Hooking CSS parsing errors to the gedit view
• Adding a lot more samples, these should range from simple examples (basic widgets in different states) to complex (basic main window sample, preferences dialogs, …)

Anyone is welcome to help, specially in the second point, as it is fairly straightforward to add new samples.

 

Just a quick reminder: The student application period for the Google Summer of Code 2011 has opened as of yesterday (Monday, the 28th of March). Apply now! The starting point for Gnome is here; it has all the relevant information.

In addition to that, if you are happen to be interested in Neuroscience and Informatics the International Neuroinformatics Coordinating Facility (INCF) also got accepted as a organization (Thanks Raphael!). Among other very interesting project ideas there are also two proposals that are Gnome related (as in pygtk based applications). If you have a cool Neuroinformatics+Gnome based idea be sure to apply at the INCF. The starting point is here.

Along the last days/weeks, I’ve gave myself the oportunity to tinker with XInput 2.1 in GTK+, now that the core concept of the involved changes seem settled (although corner cases are still being discussed).

Thanks to the previous port to XInput 2.0, handling multitouch events has been fairly easy, by enabling GDK_TOUCH_MASK events on your widget you can receive GDK_TOUCH_PRESS/MOTION/RELEASE events containing a touch ID, which is unique at that time for a touch event stream, so enabling that you can effectively receive events from simultaneous touches.

In addition, one can create GdkTouchClusters (related to a GdkWindow) and add touch IDs to it. from that point on, any update on these touch IDs stops sending individual GDK_TOUCH_MOTION events, instead sending GdkEventMultiTouch events, containing all latest events for the touch IDs in a GdkTouchCluster.

I’m pretty happy with the resulting API, althought there are some internal details left to improve, my main gripe currently is that implicit grabs in CSW still happen per device, this means no multi-widget multitouch yet within an app (also posing interesting problems with keyboard events redirection and explicit grabs). Another big item is looking into integrating this with gestures, as direct manipulation and gesturing need to go hand in hand, so that branch could be discontinued in favor of this one.

As I’m pretty lazy, and the results would be strikingly similar to what I previously posted, I won’t post any video, so you’ll have to settle for a screenshot:

This is sitting now in the xi21 branch. If you just compile this you won’t see much working though, the demo only reacts to touch events, which are only sent if XInput 2.1 is there, which requires compiling certain Xorg modules branches, and having a multitouch device at hand.

Fixing the default GNOME3 theme

Now that most of GTK+ widgets are using GtkStyleContext, I finally got onto improving the default gnome3 theme/engine, I finally chose to rip off the tiny clearlooks engine bits I needed to demonstrate how minimal engines can complement and extend the current CSS theming features. So things are getting closer to the mockups:

There are some things that still need to be improved, column headers and expanders most namely, but things should be mostly there pretty quickly. Even though, hands and eyes are most welcome on both the CSS file and the engine.

A gestures interpreter for GTK+
Despite what some might think, my day to day work carries me quite far from theming land, during the last week I got the oportunity to start development on a gestures interpreter for GTK+, a pretty interesting (read: wacky ) piece of dynamic programming, that seems to work pretty well for its builtin gestures (Video here, at the moment it handles swiping in the 4 directions, plus circular swipe in both directions)

This code lives at the moment in a GTK+ branch, including a testcase. The gestures interpreter at the moment handles individual pointer movements separatedly, although it may be improved over time to handle multiple input pointers, as the interpreter itself is private and widgets just get an enum for the gesture type.

As the stock gestures could be stored in terms of coordinates or vectors, future plans include having these stored in a mmap()-able format, plus having an editor to create new gestures.

For those building gnome3 frequently from git or paying attention to gtk-devel, this might be old news, but still worth mentioning… The gtk-style-context branch has landed in GTK+ master, which will quickly unveil a powerful theming system, powered by CSS files.

Although some apps (mostly these implementing widgets themselves) might undergo some work to adapt to this, there are API docs and migration documents (mostly courtesy of Matthias, who’s done an excellent work) that should ease the task.

Besides the GTK+ work, there is also both a gnome3 theme and engine using the new stuff. Things are still ongoing here, as many things can be leveraged into CSS, which would leave the engine very little things to do. Andreas started looking at some point into the branch, so he might have some things ready in that field .

But not only that! Carlos Garcia Campos (KaL) has been doing good progress on webkit-gtk so no backing widgets creation is needed, and Paolo Borelli has ported GtkSourceView as well, awesome!

At the GTK+ Hackfest in A Coruña I managed to get GIMP almost completely (minus one dialog and most plug-ins) running on GTK+ 3.0.

This turned out to be a great tool for finding bugs in the new GTK+. In fact, I found quite a few of them while still completing the port. Some bugs I fixed right away, others were fixed by fellow Hackfest hackers. Even while writing this post (the image was of course cropped with the ported GIMP), two more popped up and will eventually be fixed.

After being down for a while the blog is now back online. The server moved - as I did - to Munich. EOM for now - more real news later ...