Building the blog - Episode 11 - The first big bug just showed up!

Was I expecting differently?

The bad news is that I don't have anyone to blame when something goes wrong with the blog - it's my fault!. The good news is that I can always fix things myself.

Last week I stopped by the blog and noticed that only one post was showing up. The entire archive was only displaying that same post. Weird, but as any other developer, I blamed everything else but me. It was probably a hiccup with Memcache. Right? I cleared Memcache and everything worked fine.

Today I opened the blog and the same thing was happening, but this time it was a different post. Back to reality, this is my fault, not Memcache's.

Turns out that some code I added a few weeks back to cache the list of files was the culprit. I'm handling legacy URLs from my previous blog platform, and if a legacy post was accessed with an empty cache, Memcache was populated solely with that article and nothing else. From there, every subsequent request was hitting the same "list" of posts containing only one article.

Bad.

The fix was very simple, but I felt ashamed because the consequences of my bug were very nasty. Although the cache is always warm after the first visit to the blog, Google clears it from time to time, and it could have caused several readers to not see anything. I saw the issue twice over a week. Not sure how many people it actually affected.

Everything is back to normal now, and I'll make sure to check regularly over the next few days to make sure it's actually fixed. On my tests I confirmed it's not happening anymore, but one never knows.

Bottom line: It's probably your fault unless you can prove otherwise.

Lessons learned during a MySQL optimization session

One of our teams is working on a website that supports searching over a huge catalog of products. Performance wasn't doing great, and I'm all about making things fast so I jumped in to help.

Here are some new things I learned during the process. I'm also including others I knew before but I just had the opportunity of reaffirming their value:

  • It is never fast enough. No matter how much you try, there's always something else you can do to make it faster.
  • Text search on MySQL InnoDB tables is very fast compared to the same search on MyISAM tables.
  • Text search on CHAR, VARCHAR, and TEXT is case insesitive. You can read more about this here.
  • Text search on BINARY, VARBINARY, and BLOB is case sesitive. You can read more about this here.
  • The GROUP_CONCAT function is awesome, and let's you concatenate multiple rows in a single column/row result.
  • You can do full text searches using MATCH ... AGAINST. Very cool and it supports very fast natural language searches over FULLTEXT indexes. More about this here.
  • FULLTEXT indexes are only supported on MyISAM tables, or on InnoDB tables since MySQL 5.6. (The team was using MySQL 5.5 so this wasn't an option).
  • Measure everything. What can be measured, can be optimized.

I'm done with the optimization right now, but it certainly was a blast!

Having fun with CSS: Animated concentric circles

From time to time I follow some social numbers and specific statistics about this blog. Having to check different services to get the full picture is not a fun activity, so I decided to put them all together here in the blog and save myself some time.

And while at it, why not to have fun and do something different?

Animated circles to showcase different statistics.

If you haven't seen this in action, go to the About page of the blog. Assuming you are visiting this site from an HTML5 modern browser, you'll see the circles rotating, and you'll be able to navigate through all the numbers.

At the time of this post, all the stats are still not populated dynamically; that's still a work in progress. I have to connect to several APIs to get all the information, but I liked the circles so much that I couldn't wait to post this.

If you are curious, here is the complete HTML and CSS to create the circles. The pagination is using a little bit of JavaScript, and I'm also posting it below.

Let's start with the HTML:

<div id="rings">
    <div id="first-ring">
        <div id="second-ring">
            <div id="third-ring">
            </div>
        </div>
    </div>

    <div id="ring-content">
        <div class="container active" data-index="1">
            <div class="value">1,000</div>
            <div class="description">page 1</div>
        </div>
        <div class="container hidden" data-index="2">
            <div class="value">2,000</div>
            <div class="description">page 2</div>
        </div>
        <div class="container hidden" data-index="3">
            <div class="value">3,000</div>
            <div class="description">page 3</div>
        </div>
        <div class="container hidden" data-index="4">
            <div class="value">4,000</div>
            <div class="description">page 4</div>
        </div>
        <div class="container hidden" data-index="5">
            <div class="value">5,000</div>
            <div class="description">page 5</div>
        </div>

        <ol id="pagination">
            <li><a class="active" data-index="1"></a></li>
            <li><a data-index="2"></a></li>
            <li><a data-index="3"></a></li>
            <li><a data-index="4"></a></li>
            <li><a data-index="5"></a></li>
        </ol>
    </div>
</div>

There are two main sections: the first one defines the three rotating rings (first-ring, second-ring, and third-ring), and the second one defines the content and the pagination. Let's now see the CSS:

#rings {
    position: relative;
    text-align: center;
    width: 300px;
    height: 300px;
    margin:0 auto;
    -webkit-tap-highlight-color: transparent;
}

#first-ring {
    position: absolute;
    border: 5px solid #4698f2;
    border-top-color: #eee;
    margin: 0 auto;
    text-align: center;
    width: 280px;
    height: 280px;
    -webkit-border-radius: 50%;
    -moz-border-radius: 50%;
    border-radius: 50%;
    -webkit-animation: turn_clockwise 10s infinite;
    -moz-animation: turn_clockwise 10s infinite;
    animation: turn_clockwise 10s infinite;   
}

#second-ring {
    position: absolute;
    border: 10px solid #eee;
    border-top-color: #fff;
    border-bottom-color: #fff;
    width: 250px;
    height: 250px;
    margin: 5px;
    -webkit-border-radius: 50%;
    -moz-border-radius: 50%;
    border-radius: 50%;
    -webkit-animation: turn_counter_clockwise 8s infinite;
    -moz-animation: turn_counter_clockwise 8s infinite;
    animation: turn_counter_clockwise 8s infinite;  
}

#third-ring {
    position: absolute;
    border: 2px solid #315379;
    border-bottom-color: #eee;
    margin: 5px;
    width: 236px;
    height: 236px;
    -webkit-border-radius: 50%;
    -moz-border-radius: 50%;
    border-radius: 50%;
    -webkit-animation: turn_clockwise 10s infinite;
    -moz-animation: turn_clockwise 10s infinite;
    animation: turn_clockwise 10s infinite;
}		

Now to the animations:

@-webkit-keyframes turn_clockwise {
    0% {
        -webkit-transform: rotate(0deg);
    }
    100% {
        -webkit-transform: rotate(360deg);
    }
}

@-moz-keyframes turn_clockwise {
    0% {
        -moz-transform: rotate(0deg);
    }
    100% {
        -moz-transform: rotate(360deg);
    }
}

@keyframes turn_clockwise {
    0% {
        transform: rotate(0deg);
    }
    100% {
        transform: rotate(360deg);
    }
}

@-webkit-keyframes turn_counter_clockwise {
    0% {
        -webkit-transform: rotate(360deg);
    }
    100% {
        -webkit-transform: rotate(0deg);
    }
}

@-moz-keyframes turn_counter_clockwise {
    0% {
        -moz-transform: rotate(360deg);
    }
    100% {
        -moz-transform: rotate(0deg);
    }
}

@keyframes turn_counter_clockwise {
    0% {
        transform: rotate(360deg);
    }
    100% {
        transform: rotate(0deg);   
    }   
}	

Finally, the styles for the content and the pagination:

#ring-content {
    position: absolute;
    width: 230px;
    height: 230px;
    background-color: #4698f2;
    -webkit-border-radius: 50%;
    -moz-border-radius: 50%;
    border-radius: 50%;
    margin: 30px;
    cursor: pointer;
    box-shadow: inset 0 0 0 16px rgba(255,255,255,.6),0 1px 2px rgba(0,0,0,.1);
    transition: all .4s ease-in-out;
}

#ring-content:hover {
    box-shadow: inset 0 0 0 1px rgba(255,255,255,.1),0 1px 2px rgba(0,0,0,.1);
}

#ring-content .container {
    position: absolute;
    -webkit-border-radius: 50%;
    -moz-border-radius: 50%;
    border-radius: 50%;
    width: 198px;
    height: 198px;
    margin: 16px;
    background-color: #4698f2;
    transition: all .7s ease-in-out;
}

#ring-content .value {
    font-size: 50px;
    line-height: 70px;
    padding-top: 44px;
    font-weight: 700;
    color: #fff;
    text-shadow: 0 0 1px #fff,0 1px 2px rgba(0,0,0,.3);
}

#ring-content .description {
    font-size: 16px;
    line-height: 20px;
    color: #fff;
    opacity: 0;
    transition: all 1s ease-in-out .4s;
}

#ring-content .container.hidden {
    opacity: 0;
}

#ring-content .container.active,
#ring-content .container.active .description {
    opacity: 1;
}

#pagination {
    margin: 12px auto 0;
    padding: 0;
    position: absolute;
    left: 84px;
    top: 180px;
}

#pagination li {
    list-style: none;
    float: left;
    margin: 0 2px
}

#pagination li a {
    display: block;
    width: 8px;
    height: 8px;
    background: #ccc;
    -webkit-border-radius: 50%;
    -moz-border-radius: 50%;
    border-radius: 50%;
}

#pagination li a.active {
    background: #fff;
}	

And that's it. To make the information change when the content is clicked, and through the pagination, here is all the JavaScript code that's needed:

function displayPage(index) {
    var element = document.getElementsByClassName("container active")[0];
    element.className = "container hidden";

    var element = document.querySelectorAll('div.container[data-index="' + index + '"]')[0];
    element.className = "container active";

    document.querySelectorAll("#pagination a.active")[0].className = "";     
    document.querySelectorAll('#pagination a[data-index="'+index+'"]')[0]
    	.className = "active";
}

(function() {
    document.getElementById("ring-content").onclick = function() {
        var element = document.getElementsByClassName("container active")[0];
        var index = parseInt(element.dataset.index) % 5 + 1;
        displayPage(index);
    }

    var elements = document.querySelectorAll("#pagination a");
    for (i = 0; i < elements.length; i++) { 
        elements[i].onclick = function(e) {
            e.preventDefault();
            e.stopPropagation();
            displayPage(this.dataset.index);
        }
    }
})();

The code is very short, but it took me all Sunday to settle with what I really wanted. I'm happy with how it turned out, and I hope it's useful for you somehow.

The madness of layered architecture

Great post by Johannes Brodwall talking about layered architectures and why they hurt so much:

I once visited a team that had fifteen layers in their code. That is: If you wanted to display some data in the database in a web page, that data passed through 15 classes in the application. What did these layers do? Oh, nothing much. They just copied data from one object to the next.

(...) enterprise applications are not like network protocols. Every layer in most enterprise application operates at the same level of abstraction.

I've seen this so many times. I've been personally responsible for doing this most of my life as a developer. But I think I already learned my lesson.

Now every time I design something, I try to be as lean as possible. Years have taught me the value of simplicity, and I stopped a long time ago adding classes and more classes just in case we need to change this or replace that.

There's value in a layered architecture, no doubt about that. But when used incorrectly they hurt your code pretty badly; they introduce unnecessary layers of indirection that are hard to understand and maintain.

Quoting Johannes again:

Every problem in computer science can be solved by adding a layer of indirection, but most problems in software engineering can be solved by removing a misplaced layer.

Amen.

Back in 2010 I blogged about this. The article is in Spanish, but if you can read it you'll get the full picture about where I stand about this whole thing.

As a bonus, go and watch this video titled Layers Considered Harmful.

How to know if your Python code is running locally or in App Engine

This blog uses memcache to store almost everything. This is great for production, but really annoying for when I'm making changes or writing a new post.

I created a simple flag to turn memcache on and off, but I always forget to turn it back on before publishing, so most of the time I had to re-publish the code after fixing the mistake.

There had to be a better way, and it turns out that there is!

import os

is_running_locally = os.environ['SERVER_SOFTWARE'].startswith('Development')
is_running_on_appengine = os.environ['SERVER_SOFTWARE'].startswith('Google App Engine')

From the Google documentation:

The following environment variables are part of the WSGI and CGI standards, with special behavior in App Engine:

  • SERVER_SOFTWARE: In the development web server, this value is "Development/X.Y" where "X.Y" is the version of the runtime. When running on App Engine, this value is "Google App Engine/X.Y.Z".

So all I had to do was setting my flag to the appropriate value:

MEMCACHE_ENABLED = os.environ['SERVER_SOFTWARE'].startswith('Google App Engine')

This way memcache will be always disabled during development, and always enabled when deployed.

La informática en Cuba

(The last time I posted a Spanish article on this blog was back in 2011. Three years later, the readership of the blog has mainly switched to English speakers, so I apologize in advance for throwing this post in your face. I wanted to talk about the informatics on my home country, so I considered appropriate to use my native tongue as well).

Aunque no precisamente este modelo, computadoras como esta eran muy comunes a principios de siglo en Cuba. Fuente oldcomputers.net.

Hace ya una década que salí de la universidad. En junio de 2004, luego de 5 años estudiando Ingeniería Informática en la Universidad de Camagüey, llevaba para mi casa el flamante título de graduado y me preparaba para iniciar mi carrera profesional.

Mucho ha llovido desde entonces, sobre todo en el mundo de la tecnología, donde diez años son una eternidad. Sin embargo, aparentemente el tiempo no pasa con la misma velocidad en todas partes del mundo. La Cuba a la que regresé hace unas semanas sigue siendo la de aquella primavera de 2009 cuando me fui, la que me vió terminar mis estudios en el verano de 2004, la misma de siempre.

A pesar de que mucho ha cambiado, en el plano informático Cuba está sumida en un letargo. Las conversaciones de las que un día fui partícipe antes de salir siguen en boca de todos. Las mismas discusiones aún no han encontrado solución. Los viejos problemas son cada vez más problema, o quizás peor; posiblemente ya nadie quiera intentar resolverlos o todos hayan muerto del aburrimiento, o de desilusión, o un poco de ambos.

El mundo desarrollado habla hoy en día un idioma que mi país todavía no es capaz de comprender. Cuando la Internet en los países del primer mundo es algo tan natural como la electricidad o el agua potable, en Cuba sigue siendo un sueño para la mayoría de las personas y empresas. Más que la Internet, la informatización del país es extremadamente incipiente.

Basta con mirar alrededor y hacer unas pocas preguntas para darse cuenta de que aún todo es papel y lápiz. Las empresas que cuentan con computadoras carecen de personas que sepan operarlas correctamente y, cuando el personal existe, entonces falta el software adecuado.

Ocurre así a todos los niveles, desde las mismas instituciones de gobierno, pasando por empresas ricas y pobres, hasta llegar al ámbito de los nuevos “cuentapropistas” y las universidades. Peor aún, la misma -y única- institución dedicada a producir software en Camagüey me dicen que está en la misma situación: sin brújula, recursos o ideas. Si no ellos, ¿quién?

La solución no es mágica ni tiene mucho que ver con el embargo. No está en ninguna otra parte sino en el mismísimo patio cubano y sólo falta que exista una voluntad de poner la maquinaria en movimiento para que el país comience a acortar distancias.

El principal impedimento que existe en la isla hoy en día es el acceso a Internet, no sólo para los profesionales de la rama sino para todas las personas. El público crea la demanda que impulsa el desarrollo de nuevas soluciones. Sin Internet Cuba seguirá aislada y desconectada, tanto del mundo exterior como internamente, de sus mismos ciudadanos. Hay otros factores necesarios, por supuesto, y cada pieza tomará su lugar en el momento adecuado, pero lo primero es conectar al país porque la Internet es el principal motor de una revolución tecnológica.

¿Qué va a suceder una vez vencido este primer reto y luego de que la mayor parte del país tenga acceso a la red? ¿Cuánto tiempo va a tardar la informatización en alcanzar un nivel adecuado?

Los recursos humanos están, pero ¿están preparados? Me pregunto si con tantos años desconectados del mundo exterior existe hoy en día en Cuba el nivel necesario para llevar a cabo el desarrollo de la esfera tecnológica. Si nos detenemos a analizar la formación técnica de los estudiantes antes de entrar a la universidad veremos que es incipiente y en muchos lugares del país, nula. Una gran cantidad de los muchachos llegan a cursar estudios superiores sin jamás haber tocado una computadora o, peor aún, muchos se gradúan sin prestarle la más mínima atención.

El sistema de educación cubano tiene que mejorar su enfoque en ese sentido. La informática debe estudiarse desde la base para que aquellos jóvenes decididos a estudiar una de las ramas técnicas lleguen con una preparación sólida a la universidad. Si la calidad de la enseñanza aumenta en los grados inferiores veremos que se incrementará de manera exponencial a nivel profesional.

La universidad tiene, además, que adaptar sus programas al mundo actual. Es imposible mantener una carrera universitaria cambiando constantemente, pero tampoco puede estar estática por años y años sin correr el riesgo de convertirse en irrelevante. El estado del arte en materia informática cambia aceleradamente con cada año que pasa y la universidad cubana debe actualizarse y buscar vías para distribuir conocimientos relevantes a los profesionales de la rama.

Muchas asignaturas que se estudian hoy en día están desactualizadas o ya son intrascendentes para el mundo actual. Del mismo modo, temas fundamentales están siendo completamente obviados y, como consecuencia, los recién graduados tienen que enfrentarse a un universo profesional para el cual no están preparados.

La lista es larga, pero temas como el desarrollo web, las base de datos no relacionales y las metodologías ágiles deberían ser parte del programa oficial y no tópicos optativos. Asignaturas como Máquinas Computadoras, Diseño Asistido por Computadora, Gestión de Software, Automatización Industrial, Administración de Empresas e Ingeniería de Software deberían actualizar su contenido para brindar un valor superior acorde a los tiempos modernos.

Es este el momento de reformular modelos obsoletos y de impulsar a Cuba hacia el siglo XXI. La Internet será el combustible indispensable, pero la formación de profesionales jugará un papel igualmente determinante en el futuro tecnológico del país.

Es tiempo de cambios.

Mailtrap and Loggly

Here are a couple of online applications you'll appreciate if you are a developer (descriptions copied from their websites):

  • Mailtrap: A fake SMTP server for development teams to test, view and share emails sent from the development and staging environments without spamming real customers.
  • Loggly: A cloud-based log management service that helps technical teams make sense of logs that are being produced by cloud-centric applications in order to solve operational problems faster.

If you haven't heard from them before, go and check them out because changes are you already need them.