I've got a great opportunity to work with MongoDB

Personally never done it before. My whole exposure to the NoSQL world is more than two years with the Google Datastore, this book1, and a buttload of papers about a bunch of existing providers (including MongoDB).

Hands on with MongoDB? Not so much.

So this will be the time I've been waiting for (MongoDB is one of the most popular players out there. Even more than Google's Datastore2.)

I've read enough already. I've heard enough already.

Now it's time for some real action!

--

1 I've read it twice. And then skimmed through it a couple of times. Again and again.

2 A lot more.

I fucking hate dishonest sales people

Being a salesman shouldn't justify lying to a client (or a potential client). However, it seems that society sort of approves dishonesty when there's a person behind trying to sale something.

Fuck that.

I don't know how you go about your life, but every time I find somebody lying to me (or how they like to say: stretching the truth) in order to sell me something, I move on as quickly as I can.

I've been bitten already from companies that stretch their capabilities during their pitch, and burn you later on when you discover that they can't do 50% of the shit they swore they were experts at.

To be fair to those companies, there are usually two faces involved: the salesman charging commissions for every sale is made (thus pushing to sale more and more), and the people who will deliver (or won't) based on the amount of crap the salesman sold (these people usually don't make anything out of the sale but have to face the consequences of an unhappy client.)

(Sorry to all the honest salesman and saleswoman out there. Nothing at all against you. But I fucking hate people who lie for a living taking advantage of other people.)

I will never work for a company that uses tactics like this. I will never work with a company that tries this with mine.

You shouldn't either.

For when you want to stop cursing at Mr. Client for changing his mind

Of course Mr. Client doesn't stop changing his mind. This is the fourth time my team has to change everything to what we had before, after changing it again and again. It's comically stupid, demoralizing, and expensive.

Specially because he is not paying for this privilege. It's all budgeted and in the books already, but somehow he still feels entitled to make everyone work and rework again and again.

I'm sure you know what I'm talking about, don't you?

Every time I meet Mr. Client, my first reaction is to yell, get mad at him, get mad at everyone else, hit the computer, kick the dog in the butt, and keep doing stupid things until I'm exhausted. (I know people that start getting angry with the first change request, but that's a different story.)

But guess what? Getting angry has never solved the problem for me, neither for the people on my team that get to see me cranky and unmotivated. If I'm leading a team, and I let Mr. Client get to me, that's a sure catalysis for everything to fall apart.

I know better now. To help me through difficult times with Mr. Client, I put together the following checklist:

  1. Have I tried to explain the problem to Mr. Client in a professional and civilized way? Does he know the consequences for the project and the team?

  2. Have I tried to understand why Mr. Client can't settle with a particular solution? What are the strengths he is looking for? Are there any pitfalls I'm not able to see?

  3. Have I tried to find an alternative solution to discuss with Mr. Client? Something completely different to what he is trying to do?

  4. Have I tried to go above and beyond and make one of his solutions better? Have I tried hard enough to get him buy into that solution by flexing my muscles a little bit?

  5. Moving the focus to a different feature of the system usually helps. Coming back after certain time usually offers a different perspective to the problem. Can I convince him to work in something else for the time being?

Now I don't get to scream out loud until I'm absolutely sure I've gone through the entire list and done everything possible to stop the problem.

Then, and only then, when everything else failed, I start cursing again.

Why do software engineers make so much?

Do we?

From this question in Quora you can get a lot of good perspectives in what people think about software engineers, and specifically why they think that we make a lot of money.

I'm not going to say that we don't have good salaries, but I wouldn't go as far as thinking that any software engineer working for any company makes a crazy amount of money. There are a lot of variables in play, but in general is true that we are well above poverty line.

Demand is high right now and looks like it will be for a while.

Seems like a good time to be thankful.

A quick intro to NoSQL databases

Looking back at my years in college studying Computer Science, I remember quite well all the classes about disparate technologies and programming languages. It seemed that our future was going to be turbulent and full of options, but one thing remained constant throughout all the courses: we used relational databases for data storage.

Relational databases were the only choice back then. Every company used them, and every software engineer had to know the ins and outs of tables, relationships, foreign keys, and indexes to have a successful career in the software business.

But the world is rapidly changing, and with it the utter dominance of relational databases has come to an end. There’s a new kid in town, or better said, an entire family of the so-called NoSQL databases.

What’s wrong with our ol’ good relational model?

There is one thing that has drastically changed the current state of enterprise applications: the Internet. As a whole it has grown past my wildest dreams back when I was a student, to a massively global phenomenon, bringing every day more and more connected users who are producing, storing, and consuming more and more data.

Unfortunately, all this data is usually disparate, unstructured, and hard to frame into the rigid schema constraints imposed by relational databases. On top of that, accessing this data when structured in a relational way requires collecting bits and pieces from multiple tables to conform the final result presented to the user. This process is slow, and although it works well for medium-size applications, with so many new users and so much more data, the performance implications are amplified to the point where alternative solutions are needed.

When faced with these scalability problems, engineers used to vertically scale their infrastructure. Vertical-scaling means adding more hardware to the problem: the database server would get a better and faster CPU, more memory, and more hard drive space. As you might have guessed already, this type of scaling is not only limited, but also extremely expensive.

But the internet doesn’t care and keeps growing. And more and more users are online accessing our application and producing more and more data. And at certain point, our database is the single bottleneck of the entire architecture. Setting up a server farm is a complicated process, and replication models to horizontally scale a relational database require too much money, too many trade-offs, and too much time from a team.

Making horizontal scaling a possibility

If instead of having one super-powerful database server attending our increasing number of users, we have multiple regular machines sharing the load we would be able to scale indefinitely for a fraction of the price, right?

Right. But this means that we’d need to relax some of the schema constraints imposed by relational databases in order to easily shard our data into multiple partitions distributed in multiple servers. Breaking apart a strong relational database into several chunks of data is not an easy task, especially because the existing intricate web of relationships and constraints. What makes relational databases great, is also their main drawback for scalability purposes.

Here is where NoSQL databases have something to offer. Among all their specific characteristics, they usually don’t impose a fixed schema for your data, and you can structure them in a simple way for the database motor to partition it into multiple shards. This flexibility allows engineers to very easily map any piece of information without worrying where that data will live or how it will be distributed.

Of course, this comes at a price; NoSQL databases lack all the structure that make relational databases easy to use and understand. This lack of structure often leads to duplication of the data, and to a less expressive query language. Like any other problem in computer science, whether we use NoSQL or stick with our relational database is all about context.

The NoSQL database family

If you are reading this article, you are probably familiar with relational databases. No matter what vendor is behind your preferred engine, they all have a lot in common. The steps you followed yesterday to model your database are probably the same steps you’ll take tomorrow. Relational databases are very well known and their similitudes make working with them and moving from one to another a very straightforward process.

On the other hand, the NoSQL family is huge with 150 different databases at the time of this writing. These databases are categorized in different groups based on their main characteristics and features. There's certainly something out there for every specific need that you might encounter with your particular application.

Why should you care about NoSQL?

There are multiple reasons, but I'm going to mention the main three from my point of view:

  1. Is where the market is going. Take a look at some of the job posts from big technological companies and you'll see how demand for NoSQL database is on the rise, and with more data being produced every day, the NoSQL family will become more and more useful.

  2. Scalability for your own application. If you are building an application (or are planning to) and you are struggling with the idea of a massive amount of information, start taking a look into the NoSQL world.

  3. It will take you to a different dimension of Computer Science. If you are up to increase your value as a professional, pick your tool and dive in. If your experience is anything like mine, you'll be for an unforgettable ride into all sort of technical challenges.

It will be fun for sure. If you haven't already, find the opportunity and start learning.

The browser's back button

Writing a Single Page Application? Then you probably imagine what I'm about to say: click on the browser's back button and make sure it really takes you back. If it doesn't, then you still have work to do.

Because it's a single page (duh!), everything is built showing and hiding different sections inside that page. This is why clicking on the back button usually takes you off the site and not to where you expect.

Which is confusing for users. We developers know what's going on, and we (of course) try every possible trick to avoid fixing it, but for users of the site, it's really clunky clicking on the back button to land anywhere but where they were expecting.

Fixing the problem

Here are some resources for you to take a look:

Unfortunately there's no simple switch to make the app magically work (although some JavsScript framerworks make it considerably easier than others.) There's a litle bit of work involved, but I'm sure your application deserves it.

Here is a post you won't believe I wrote until you read it

Just for the fun of it, and following the same criticism that others started (see Signal v. Noise's or Scott Hanselman's posts), here are some new titles for 5 past posts I published on this blog:

  1. Something you'll be surprised I did despite how expensive it was

  2. It is unbelievable how stupid is this guy!

  3. The only one secret to get a job that people keep forgetting about

  4. You'll be shocked to know what this huge company is capable of doing!

  5. The only one thing you need to know about software that will change your career

Yeah I know. Sorry about that.

False advertising

Lately I've been thinking a lot about false advertising. It makes me angry to see how many people (including myself) get caught in the spider web created by individuals trained to make us believe what is not necessarily true.

Nobody wants to tell you what the real cost is. Nobody wants to give you all the information you need to make a good decision. It's all about another sale, another lead. It's all about feeding The Machine with you, the ultimate product.

And I also realized that software developers are good at this. In a different context maybe, but based on the same principle: we lead our clients to make decisions that we like, not necessarily the best ones:

  1. To work on the features that we are most excited about.
  2. To not fix the issues that we don't fully understand.
  3. To discard those features for which our code is not ready.
  4. To take the path that's simpler for us.

And of course, this is not helping the products we are working on; it only feeds our own satisfaction.

Remember we were hired to help. We were hired by someone to read and write the code, to somehow translate a foreign dialect to business words. If we can't be objective, if we can't have a conversation without thinking about all the fine print, then we aren't better than the rest.

And I know it's hard to be objective when's us in the middle, but we ought to try harder.

(Some people do actually mislead others on purpose. For those, remember that you aren't the only person good enough to read and write "the language". Sooner rather than later they will find out your tricks. And then you'll be in disgrace.)

Small steps aren't necessarily the right way

A bunch of agile methodologies focus a lot on "incremental changes", and "the minimum thing that gets us there". A lot of people recommend moving forward "a small step" at the time, and others always say to "forget about the future" and concentrate only in the next step.

That's all great, but dangerous: a bunch of small optimal changes to a software system may turn the overall architecture into a chaotic mess.

Every day you make hundreds of small decisions related to the code of your system, and although each one of them might be the right short term move, you may be destroying the long term design and architecture of the application.

Developing software is darn hard and you probably don't need me to try and make it harder for you, but truth is that you can't just jump ahead and bit by bit peel away whatever sanity is left in your code.

Always keep the long term vision in mind: Is this change going to take us closer to our goal? How does it fit with the existing architecture of our application?

One step at a time. The right way.

Improving validation with adaptive messages

Here is a very sounding post about how to create better validation messages. I agree with Jamie: there are error messages that absolutely suck, and we should do better if we want to retain users.

Form validation errors are inevitable. Yes, they can (and should) be minimized, but validation errors won’t ever be eliminated – they are a natural part of complex forms and user’s data input. The key question then is how to make it easy for the user to recover from form errors.

I read the entire post and liked it very much. However, I wanted to add a couple of caveats:

1. Return of Investment

Before jumping and start creating variations for our error messages, we need to understand what's the ROI for each one of the problems with are trying to solve.

For any given field, we can easily find a million ways to input the information that goes in it in the wrong way. Does it make sense to create a specific message for each incorrect variant? Of course not. We need to identify which are the most common ones and only implement those.

But even before, we need to think about what Jamie said and I quoted above: error messages should be minimized, which leads me to the next point:

2. Do, don't tell

Should we spend our time working on specific validation messages, or should we rather spend it working on avoiding error messages on the first place?

I'd like to think that if we are good enough to detect that something specific is wrong, we are probably good enough to fix it for the user.

So instead of detecting specific problems to display errors to the user, we can spend the time trying to recover from these problems without having to display the messages at all.

Let's take the US phone number example from Jamie's post. Here are some possible erroneous inputs (think that only a 10 digit number is a valid input. For example: 4343433423):

  1. 434-343-3423: User entered dashes. Instead of displaying an error, we can remove them and save the number as 4343433423.

  2. 434 343 3423: User entered spaces. Instead of displaying an error, we can remove them and save the number as 4343433423.

  3. +14343433423: User entered a country code (+1). Instead of displaying an error, we can remove it and save the number as 4343433423.

  4. 434343342: Here the user is missing a digit. In this case we don't have any other choice than to display a specific error message.

As you can see, from 3 out of 4 possible errors we can easily recover. In these cases, we don't need to display the adaptive error message and we'll be providing a better experience overall.

Always think before doing

Like with anything else, there's no right or wrong way of doing things. Every application faces different constraints, and your ability to tackle the right problems in your specific context is what's going to make you a great engineer (or a crappy one).

Following Jamie's advise will add a nice touch to your application, something that you'll be able to measure in real dollars based on retention rates and fewer frustrated users.

But it comes at a price, and it's up to you to decide whether is worth it.