Introduction

This document explains some best practices for application error/debug logging. Application logging is an important “tool” to help identifying problems. If done correctly, and if an infrastructure is in place that allows events to be put into context, it’s easier to debug situations and thus makes it easier to solve problems sooner.

While out of scope for this document, I’d like to point out the following tools that can make that possible:

• Logstash – http://logstash.net
• Graylog2 – http://graylog2.org

Different logging meanings

Logging can mean different things and logging can have different purposes. The logging this document refers to is application and semantic logging.

Application logging (also called “debug” or “error” logging) is intended to keep track of the “low level” application behaviour. Application logging is used to log events, that allows for debugging a problem within an application or to monitor application health. In contrast, a different form of logging could for example be Google Analytics. Tracking (or “logging” actions of) visitors of a website, this type of logging is very different and not something this document refers too.

Why do we log

In order to understand how to log, it must first be clear what we try to accomplish with application logging.

Continue reading “Logging best practices”

Imagine a world wide web without cookies. And this might not be the strangest thing, since the use of cookies is severely limited since Wednesday the 22nd of June 2011 in the Netherlands. And more countries will follow: http://www.bbc.co.uk/news/technology-12668552. 

The Law 

The (Dutch) law, that requires a user to agree before storing data, doesn’t only apply on HTTP cookies. But in fact any kind of data that is stored on the users computer. Such as; HTML5 storage, flash cookies. But also desktop applications, etc.. The law also states that cookies “required” for certain functionality, are allowed without confirmation. Personally I don’t see how anything will change, with this exception in place. And I wonder how many experts were involved into making this law. But that is a subject for another article perhaps…

What are cookies

Cookies are little packages of information stored in the browser of a website visitor, they can contain “small” amounts of data such as an identifiable token or a user preference.

What purpose do cookies serve

Cookies are very generic and can be used for many things, good and bad. The most popular probably being tracking your activity and advertisement. But they are also used to keep a state between requests and to store a preference. Such as “remember me” at a login form, or perhaps “no I do not want to participate in your survey”.

Cookie problems

If When the law becomes international, you are potentially violating laws by simply calling something like: setcookie() without the users consent. There are hacks around the typical HTTP cookies, by using flash cookies for example (Oh and btw, here a guide on how to clear those: http://www.macromedia.com/…ngs_manager07.html). And possibly HTML5 storage could play a role here (or any of it’s derivatives).

Another thing that has been happening, is visitor awareness and thus browser features. More and more people block cookies to stop advertisement tracking, but unfortunately this also prevents a user to use the features he or she wants to use (such as login sessions, etc.). There is an answer for this and quite a few browser vendor’s plan on implementing the “Do Not Track” (http://donottrack.us/) feature, or have already done so. But I’m not too happy with it. The downside of “Do Not Track” is that it’s voluntary for website owners and advertisement companies to respect this feature. Other tools include projects such as “Ad Blocker”, that only block cookies (and more) for advertisement purposes. It works pretty good, but that is hardly user-friendly.

But, back to “no more cookies”… How do you solve the problem of keeping a state between requests over a stateless protocol?

Some ideas 

A couple of years ago, when I joined Ibuildings, it was because I had a passion for PHP and the PHP community, and Ibuildings was THE company embodying this spirit. I’ve got the opportunity to work at some of the most amazing projects, and met some of the smartest people in PHP-land. However since about a year, I’ve noticed my interests shifting a lot to mobile and mobile development. Of course I’m still as passionate about PHP as I was a few years ago, but I feel that mobile development is quite close to web development in a couple of ways, especially nowadays when people often expect to have internet available at all times. After focussing more and more on mobile development and seeing a huge future in it, I decided it was time for me to move on and leave Ibuildings.

To better match my combined interest of PHP and mobile development, I decided to join Egeniq. For those who don’t know Egeniq, they’re an awesome mobile technology company, founded 7 months ago by my former colleagues Peter, Bas and Ivo. Other than your average mobile app shop, they focus on proper software engineering, architecture, performance and scalability. They build on the years of experience they have in the PHP world and bring that expertise to the mobile business. They’re also really into geek stuff, and mobile devices fit nicely into that category, too

Even though deciding on leaving Ibuildings was probably the hardest career decision I ever had to make, I very much look forward playing around with all things mobile and working closely with my former colleagues again. Exciting!  I’d like to thank Ibuildings for the opportunities they gave me and all the wonderful people over there, from whom I’ve learned so much. I will be at Ibuildings until the end of April, starting at Egeniq at the start of May.

This is quite a big step, and I thought about taking this step for a long time. But you know you should pursue something, if you have a very strong gut feeling that you just need to do this. The feeling that, if you don’t do this, you will regret it. The feeling that you get when considering all the pros and cons, but deep down inside yourself you know you already made up your mind. It goes without saying that I’m very much looking forward to the new things to come at Egeniq!

Reverting a dropped column

In my talks and article I always mention that when it comes to database migrations it is generally a bad idea to rely on undo-patches for rollbacks. As an example I always use the same story: “Imagine you write a database patch that removes a column from a table, and then you write an undo-patch that adds the column back again. Sure, your database schema is now the same as it was before – but the content is gone!” And then I would move on by saying that you probably shouldn’t rely on undo-patches anyway for rollbacks, because it is better to thoroughly test everything before updating so you are absolutely certain that you won’t have to rollback. Also, I recommended to make backups first, and use that to revert database migrations when needed. Easier said than done!

The fact that I am in the luxurious position where this approach usually is an option, doesn’t mean that this is a solution for everyone. For other projects where, for example, larger databases are involved, rolling back by restoring a backup would be considered highly inefficient (and if the database is in use it would actually result in data loss). Being able to rollback patches can be convenient, but reverting dropped columns can’t be done off course.

When I did my talk at DPC10 in Amsterdam however, someone in the audience (sorry, I don’t remember who it was) suggested a very simple but very effective solution. In fact, it was such a simple solution that I couldn’t stand I hadn’t thought of it before: If you rename your column or table first instead of dropping it you can still just use an undo-patch to revert the change. In a later stage, when you are absolutely certain you don’t need to do a rollback anymore, you can simply drop the renamed column or table by writing another patch. For example in a next release.

Reverting a renamed column

As you can see, sometimes the simplest solution is the best. Beware though, as in some situations this approach might still cause problems: Records added to this table between the moment the patch was applied and the moment the undo file was applied won’t have a value in the age column, to name just one. As always when we are talking database version control, it all depends on the specific situation you are in.

I myself still refuse to write undo-patches as most patches create new tables or columns anyway, instead of removing them – and if patches actually do drop tables or columns they probably haven’t been very significant for quite some time anyway. Maybe the main reason is that I have simply never found myself in the situation where an undo-patch would have helped me. This might be different for you though, and if you are looking for a way to rollback dropped columns or tables, renaming them first might be a pretty good idea.

A very popular and widely used PHP extension for debugging and profiling PHP applications is Xdebug. Normally this tool would be used on your development environment, or in rare occasions on some remote environment that is similar to the production environment. However, a little while ago one of my co-workers found a production environment running Xdebug, and we wondered how severe this was influencing performance. Since you are adding extra overhead it is very likely things will run slower. But how significant is this difference. Would it be something worth considering?

Disclaimer
Now I know that debugging a production environment is generally considered unnecessary and a really bad idea. I myself can’t really think of a situation where I would actually want to have Xdebug on my production machine – but that’s not the point. I just had a new question that I wanted to answer: “How much does installing Xdebug affect the performance of PHP?”

The hardware
To be able to test this I first needed to find a proper testing environment. Unfortunately I don’t have any webservers laying around, and testing directly on my macbook would probably be a bad idea – since there are a million processes running that all influence the performance of the machine, and therefore would influence the results. After some pondering I realised that I had an old (well… more like ancient) laptop laying around that would actually suit this purpose quite well. When it comes to speed it doesn’t come anywhere close to a modern production environment – but since we will be looking at relative changes rather than absolute measurements I figured it would be just fine.

The software
I downloaded and installed a fresh new copy of Debian. Once it was installed I decided to take the easy way and install PHP simply by using apt-get install php5. Finally, I installed xdebug using the PECL installer. With just this installed and nothing else, I was convinced my benchmark results would be quite reliable.

Software installed:

• Debian 5.0.8
• PHP 5.2.6 with Suhosin-patch
• Xdebug 2.1.0

The benchmark
Why write a benchmark when they already exist? For testing purposes I decided to use the bench.php script that comes with the PHP source. This script does not run out-of-the-box when Xdebug is installed, because one test (Ackermann function) exceeds the default Xdebug limit of 100 nested function calls. Now I could of course increase this number, but I wanted to test how Xdebug performs without any modifications to the configuration – so I decided to simply skip this one test and just run the remaining ones.

Finally, after a lot of preparation, I was ready to run the script. \o/

The results
I decided to run the script three times with the xdebug extension enabled, and then another three times with the xdebug extension disabled. The results were pretty clear:

So there we go. Without Xdebug it takes about 18 seconds to run the script, and with Xdebug enabled is takes about 44 seconds: 241% of the original time (or: it took 2.41 times as long to run the script).

So what does this mean?
In this situation adding Xdebug slowed down the execution of the benchmark script quite significantly.

So what doesn’t this mean?
This doesn’t mean that your application will slow down with the same relative percentages, nor does it mean that you can compensate for this by buying 2.41x more servers. When we look at the breakdown per test it’s clear to see that the differences per test are quite significant as well. For example: the time needed to run the ‘ary’-test increased with only 27%, while the time needed for the ‘simplecall’ test increased with a whopping 379%.

Apparently the overhead caused by running Xdebug varies and is related to what kind of operations you are running. Calling functions seems to be affected a lot by Xdebug, while the effect of manipulating strings or arrays is less severe.

A more realistic setup – Testing WordPress with ApacheBench
The numbers from the previous test are quite clear – but they hardly represent a real world example of an application. To get a bit more insight in how enabling or disabling Xdebug would affect an actual real life application I decided to add Apache, MySQL and WordPress into the mix, and benchmark the default homepage of a standard WordPress installation using the benchmark tool ApacheBench (ab), which comes with Apache. Adding these applications means there will be a lot more factors to influence the results – but I was hoping the results would somehow give an indication on how much an average application would be affected by installing Xdebug.

I ran ApacheBench a number of times. Every time it fired a total of 100 requests and every time I changed the number of maximum concurrent requests. I assumed that when Xdebug was enabled the requests would take longer – but because of that it’s also more likely the maximum number of concurrent requests that’s still acceptable is reached sooner. After a couple of test runs I had the following figures:

or in a pretty graph:

So not only does it take about 20% more time to respond to requests in normal situations, the maximum number of requests your server can handle is also influenced significantly.

Conclusion
So as a conclusion we can say what we actually already suspected, Xdebug influences performance quite a lot. How much exactly is hard to tell and depends on what your application does. Xdebug is a great tool, and pretty much irreplaceable when you are looking for a proper tool to debug or profile an application on your development- or testing environment. Running it on your production machines however is something that would only make sense in very specific situations, as it is typically unnecessary and it slows down your application a lot. Also, make sure you turn off Xdebug on environments that you want to use for benchmarking your application – as it will influence your results.

Edit: Directly after publishing this article Xdebug mentioned on twitter that the current version in SVN is supposed to be a lot faster. I’ll benchmark this new version soon – and let you know if there is any significant different.

Last year I spoke at different conferences throughout Europe about database version control. However, a while ago I decided that I did the talk often enough and that it’s time to move on. Therefor I wrote a big wrap-up article that summarizes everything I told (and learned) during these events. I’m proud to announce that this article was published on ibuildings’ techPortal site today!

You can find the article here:
http://techportal.ibuildings.com/2011/01/11/database-version-control/

As mentioned before I’ve done my talk “database version control without pain” a couple of times last year. When I did my talk at PHPNW10 in Manchester the organisation made videos of the different speakers, and last week they published the video of me doing my talk on blip.tv.

I found it quite shocking to see myself doing my talk. I’m sure we’ve all been there: you record your own voice (for example on you answering machine) and when you hear it back it sounds really awkward. It’s that feeling, but then with video… in a foreign language… 1 hour long. Anyway, I learned a lot from seeing this back, and I can definitely use the video to improve my future talks. For example: I now see I really need to be more aware of where I keep my hands when I’m speaking, and that my English still has this annoying dutch accent going on

Anyway, if you missed the talk or if you want to see it again, or if you simply want to see me speak in front of an audience (hi mom!) this is your chance!

View the video on blip.tv

The slides hard to read on the video because of the low picture quality. Therefor I’d recommend keeping the associated slides next to it and click along with me.

Many thanks to PHPNW, Magma Digital and everybody else involved in organizing this great conference and making and publishing the videos.

Whenever I do a talk, it always surprises me how many people ask about my slides. Some people couldn’t make it to the conference and ask for the slides instead, others want to have the slides as a reminder of the talk and some of them might even use the slides to do a slide karaoke back at the office. Great purposes, and usually I’m more than happy to share them with the world. What surprised me even more however, is that some people seem to get upset whenever a speaker decides to not share his slides – or simply forgets about uploading them, or hasn’t come round to doing it just yet. “Whenever a speaker doesn’t upload his slides to slideshare, God kills a kitten”, seems to be the general thought. I disagree for a number of reasons.

My talk is more than just my slides

Photo by Jeroen van Sluijs

First of all, talks are usually more than just the slides. For example: during the most important part of my last talk, my slides showed a picture of a slice of pizza for about 3 minutes. By browsing through someones slides you might get a general idea on what the talk is about, but you’re missing out on the details – and probably the most important message the speaker is trying to get across. It gets even better when the speaker uses more “Presentation Zen”-like techniques. Thijs Feryn‘s talk “PHP through the eyes of a hoster” is a fantastic talk, but if you would download his slides you would mainly just end up with a bunch of beautiful, but meaningless photographs (and some keywords).

That however is the way slides should look! They should confirm and strengthen what the speaker is telling, not tell the story themselves. Therefore sharing your slides then is not only pointless, but whenever somebody is going to see your slides it’s likely that the viewer will get a crooked view of what you were trying to tell in the first place.

Sharing slides is a gesture
I share them however, and most speakers do. My main reason is that people can browse through the slides again afterwards, and remember what I told them. If I did a good job the pizza-slide will then remind the viewer of the used metaphor and what I was trying to tell when this slide was on the screen. Even people that didn’t see my talk might still get a clue about the contents by looking at the slides. They will probably miss the main clue, but at least they will see a couple of mentioned tools that are worth giving a try – and if that makes you happy that’s just great! In such a case I’m glad you at least found some benefit in the work I’ve done.

Sharing slides is a gesture though. Something extra the speaker does especially for you. Not sharing slides is not “evil”, it’s normal. There are plenty of reasons why a speaker wouldn’t upload his slides. Maybe he thinks the slides themselves shouldn’t be viewed because the viewer would miss out on so much background information and explanations that it makes the talk look plain and stupid. Maybe there’s a reason like copyright restrictions on used photographs, or maybe the speaker doesn’t want to share his slides because he wants to do the same talk somewhere else next month and he doesn’t like it when people in the audience are already reading his slides before he has even started the talk.

Either way, it’s the speaker’s choice and not something that should be taken for granted. Sharing slides is a little bit extra a speaker did for you, and worth saying “thank you” for. Speakers don’t owe you anything, they usually don’t get paid (worse: they often even have to pay for their own expenses just to be at the conference) and they’ve usually put many hours into preparing their talks. All just for you! So next time, be grateful for the efforts they’ve put into it and give them a beer, instead of bitching about speakers that for whatever reason chose not to share their intellectual property with the entire world.

About a year ago the idea was born to do a talk on database version control. Main reason: I didn’t really have a clue about database version control myself. What’s the right approach? What tools are out there? Am I doing database version control the right way, or is there a better way? While figuring all this out I decided to document all my steps, and finally use the results in a talk so I could share them with the rest of the world. I submitted a talk called “Database version control without pain” to the dutch conference PFCongres, and it got accepted!

Zwolle, Amsterdam, Manchester and Barcelona

Soon after that I got to repeat the same talk at other places in Europe. During these events I met some great people, had a lot of fun, saw cities I had never been before and I learned a lot as well. I tried to improve my talk based on the feedback I got, and sure enough most people seemed to like it!

In total, I did the talk four times now:

• PFCongres, Zwolle (The Netherlands)
• DPC, Amsterdam (The Netherlands)
• PHPNW10, Manchester (UK)
• PHP Barcelona, Barcelona (Spain)

Enough!

I think four is enough. I assume that the majority of the European PHP community has visited at least one of the events above, and had the chance to see the talk (and besides that I’m getting a bit tired of repeating the same story over and over again ). It’s time for something new! I don’t exactly know what it will be, but I have several ideas on new talks so hopefully I can make at least one of those ideas into a talk, and start submitting to the calls for papers at the different conferences really soon!

Article on database version control

As a last hoorah I’m planning to write an article on database version control, which will probably be published either here or on techportal. Missed the talk? Looking to refresh your memory? Or downloaded the slides and you’re still wondering what the heck I was talking about during the slide with the picture of a sunken boat? No worries! Just keep an eye on this site and you’ll be able to read all about it soon.

But for now: Farewell database version control talk! You’ve been a great friend!

It has been a while, but here is part 4 of the PHP Quiz series! A few questions to crack your brain about, or perhaps you know them all? Try them and find out! Also do read the idea behind these quizzes, here: The PHP Quiz series

As always, think of the answer before you execute the code or look it up. Codepad might help you run the examples. You can find round three here.

Visibility is key

Now you see me, now you don’t

Static, sticky, icky

Getting the class

The strptime function

The oldtimer