Marco Troisi | technology, software development and productivity

What Makes Developers Productive? Four Observations

Sat, 30 Mar 2024 13:00:41 +0000

I have been a technologist for well over a decade, working in roles such as software engineer, software architect, team lead, and most recently, CTO of a small startup.

While I’ve certainly had to deal with the more managerial sides of some of these roles, it’s fair to say that I’ve been decidedly on the Maker side of the Maker vs Manager equation.

Even as a CTO, my time and tolerance for politics and meetings has been fairly low.

It’s been a lifelong passion of mine to figure out just what exactly does it mean to be productive. Over the years, I’ve incessantly asked myself the question: what can I do to make the most of my time and add lasting value?

As a young and fairly junior software engineer, that question seemed like a fun one to answer. Beginning to understand the answer to it, though, has helped me immensely to make the leap forward with my career and grow my skillset¹.

What follows are my observations on productivity. After countless conversations with individuals in the industry and creatives/makers in general, I believe that these observations apply to many, if not most, of us.

Deep Focus is Key: To produce my best work, I need long stretches of uninterrupted time.
The Cost of Context Switching: Interruptions aren’t just time-wasters. The mental effort to refocus after an interruption (the “switching cost”) can derail an entire day.
Urgency is Rare: The vast majority of interruptions are not urgent and could be handled asynchronously.
Misaligned Systems: Many workplaces unintentionally undermine productivity with chaotic open offices and tools that encourage constant communication (like Slack).

Why is observation #4 the case? It’s hard to know for sure.

One theory is that most of these tools and systems are chosen by the people in charge. These people, almost inevitably, fall into the Manager camp. They thrive on meetings, and can’t fathom why anyone would need quiet to get work done. They equate real time communication and meetings with “collaboration”. That makes anyone asking not to be interrupted the bad guy! Why would you not want to be collaborative and a team player?

Another theory is that we don’t have (or don’t know how to use) the right tools. These tools should, minimally, facilitate the following:

Visibility should be clear. At a glance, it should be obvious what everyone is working on. This provides managers or team leads with the necessary assurance that work is progressing well, eliminating the need for frequent status meetings or a barrage of “How’s the project coming along?” Slack messages.
Communication should be effective, allowing for the exchange of information and requests for help. Messages should be read and responded to without the need to interrupt the other person unless there is a truly urgent matter, which is often rare.

Attempts to tame our existing tools have been made. Many organizations have rules or best practices for using Slack. For instance, some companies clarify that immediate responses should not be expected when sending messages. While this usually helps, there’s a recurring observation that delays in responses on Slack can induce anxiety. It’s almost as if the tool itself is unfit for purpose, but that’s another complex issue we’ll not delve into right now!

So, here they are, my four observations about productivity. As mentioned earlier, they may not be easily achievable. However, understanding what works and what doesn’t, along with the conditions under which our brains function best, can make a significant difference.

Is the current state of work truly optimised for deep focus and uninterrupted creation? Let’s keep this conversation going. Let me know your thoughts on how we can better bridge the gap between how developers work best and how workplaces are often structured.

[1] I never did imagine how much more relevant the productivity question would become as our family grew. When you have small children, your available time becomes ever more limited and catching up after work hours is often not a viable option!

This article was first published on The Serverless Mindset.

Simplify your microservices security with these cloud services

Mon, 14 Jun 2021 08:07:41 +0000

I recently wrote an article for TechBeacon where, among other things, I made the case that microservices security can be enhanced and made easier by adopting a “serverless-first” approach.

If security is important to you but you have neither time nor desire to turn it into your (or somebody else’s in the team) primary concern, then you should consider adopting at least some of the services and approaches that I suggest in the article.

You can read the full article here:

https://techbeacon.com/security/simplify-your-microservices-security-these-cloud-services

The key to chatbot success: High-quality conversations

Thu, 09 Apr 2020 08:07:41 +0000

When you’re building a chatbot or virtual assistant, the quality of the conversation should be the most important consideration. A chatbot should adopt features and characteristics that deliver a high-caliber conversational experience.

You can find resources that describe the many advantages of using chatbots for business, but the conversational aspect never seems to receive the attention it should. Run a quick web search for “advantages” or “benefits” of using a chatbot, and here is what you’ll find:

It’s available ²⁴⁄₇.
It scales.
It helps you save money.
It automates your work.
It cuts down on human error.

While all of those attributes are very useful and are definitely part of what a bot can help a business with, they are not unique to bots.

Understanding that human-to-human interaction is the most effective way to communicate and get things done is key to building successful chatbots and virtual assistants.

What does it mean to be ‘conversational’?

What exactly does it mean for a bot to be conversational?

In their paper “Perspectives for Evaluating Conversational AI,” researchers Mahipal Jadeja and Neelanshi Varia identify the characteristics that people generally expect of a human assistant; these features are what people expect to see in a high-quality virtual assistant or chatbot.

Knowledge about the user—This includes things such as the time of the day the user typically prefers to set meetings and whether she is more of a tea or a coffee person.
Problem solving and influencing skills—A good personal assistant knows his boss (the user), understands her mood, and can come up with appropriate words to avoid conflict and offer some relief during a stressful day/season.
Security and trust—Confidentiality and a sense of mutual trust are maintained at all costs.
Efficient organization—The personal assistant should have relevant files, documents, and so on, all well organized and readily available.
An understanding of the job—The personal assistant should understand the rules and methodologies the company has adopted.

This is a great place to start. Knowing that the above features are what’s expected of a good personal assistant, you can build your virtual assistant while trying to perform as highly as possible in these areas. (Another good resource is the book Designing Bots: Creating Conversational Experiences.)

You can also look at a list of skills that a virtual assistant using natural language should have. You can watch this video to see how Gillian McCann, head of cloud engineering and AI at natural-language platform provider WorkGrid, defined a “good” natural-language assistant at the 2019 Amazon re:Invent conference. (McCann starts speaking at around 34 minutes into the video.)

According to McCann, a conversational assistant should be able to:

Perform tasks, including specific actions such as scheduling a meeting or creating a ticket. These are likely to be API calls to third-party systems.
Provide intelligence. This means personalization, such as offering personalized answers based on where a user is located. It also means memory—that is, the ability to provide and retain context about the user.
Provide entertainment. The conversation should feel unique, engaging, and interesting.
Have knowledge. This is the most important element. Ultimately, your chatbot or virtual assistant should be able to answer specific questions with clear, conversational answers.

On the last item, McCann later offers a good mantra to help make your chatbots knowledgeable in a conversational fashion. She calls it “answers, not links.” For a regular web search it is acceptable to respond with web links, but that’s not the case when it comes to a conversation.

When you ask someone a question, you expect a reply that involves at least a few words! Of course, you could also provide a web link as a plus, but your user will expect more than that.

Measuring your bot’s conversational skills

So far, you have learned about some of the characteristics of a human personal assistant, as well as the expected capabilities of a good-quality natural-language assistant.

But how do you know if your bot is both conversational and effective? Researchers Jadeja and Varia suggest a few universal metrics that everyone can adopt.

Compare your bot’s interactions with those of a live human

This metric, called “user perspective,” asks you to try to execute the same task that you built your chatbot to do, but with the help of a human assistant instead of the chatbot. When running this metric, observe the quality of the interaction between user and assistant.

This is the most critical evaluation metric you can run, but it’s also the most expensive, because it involves relatively specialized humans in the process. The metric is crucial because it gives you a better understanding of the user’s expectations, lets you establish trust between the user and the virtual assistant, and helps you to understand the tactics and methodologies adopted by the user to get the job done.

Measure the information retrieval

Here, you want to evaluate whether a bot is able to find and display the information the user had requested. This is called an information retrieval (IR) perspective.

While this metric is very important, it doesn’t give you the full picture. For example, a bot may show a list of products the user is looking to buy, but still miss the mark because of some nuance in the way the user asked for it—something that a human assistant would have picked up on—that indicated an interest in a specific variation of those products.

This metric also isn’t particularly helpful when it comes to the way the virtual assistant interacts with the user. In other words, this metric may prove that you have a sophisticated search engine behind the scenes that can find anything the user asks for, but it says very little about the way the information is presented back to the user.

Nevertheless, as shown in the features and skills listed above, knowledge (the ability to answer the user’s requests with useful results) is essential to a good virtual assistant.

Measure linguistic properties

H.P. Grice, in his book Studies in the Way of Words, outlines the four principles required to achieve maximum cooperation between people having a conversation. These are explained in more detail in the chapter titled “Logic and Conversation,” printed in full here (PDF).

Quality: What is being said should be true and evidence-based.
Quantity: The amount of information shared should not be too much or too little.
Relation: The response must be related to the conversation.
Manner: There should be no ambiguity, and the interaction should be direct and straightforward.

This metric requires judgment by a linguistics expert. As Jadeja and Varia put it, “Who will decide whether the conversational AI’s response is related to the topic or not?” There’s no standard way to determine that. Still, looking at Grice’s four principals can be very effective for improving the level of trust between the user and the virtual assistant.*

Run a standard AI measurement

The most well-known AI measurement is the Turing test. The drawback to this approach is that it doesn’t give you any recommendations for how to improve your bot’s conversational skills.

It may give you a one-off assessment as to how well or badly your conversational AI system is doing, but it won’t give you any direction on how to make it better. This is a great starting point though, so don’t discount it.

How to get started

The above, in a nutshell, are the characteristics, features, and skills your bot should display. Some of these are things that you would expect of a human assistant, while others are more specific to virtual assistants (e.g., being able to call an API to provide answers).

By combining these features you’ll make a great chatbot that gives converses with your users a the highest level of quality.

Also, try running the universal metrics described above to determine how well you’re doing. These measurements will help you discover whether your chatbot is truly conversational. This is what will set your virtual assistant apart, since it allows you to deliver an experience that goes far beyond what the simple web pages or online forms that your competitors may be using can provide.

Have you built a chatbot or a conversational interface? What’s your experience with it? Leave your impressions in the comments below.

This article was first published on TechBeacon.

Why AWS

Sun, 28 Jul 2019 08:07:41 +0000

One of the most common questions I receive when talking about building an entirely serverless platform on the AWS cloud is “but why AWS?”.

This is a question that usually comes with some assumptions. For example, some people believe that other cloud providers are better, and so they wonder what made us choose AWS over those. By and large, though, the main assumption seems to be that so called ‘vendor lock-in’ is a bad thing.

My answer typically comes in the form a few arguments:

Vendor lock-in is not a bad evil
The best way to benefit from the cloud is to go all in
AWS is the best cloud provider out there

I will try to walk through these points in this post.

1. Vendor lock-in is not a bad evil

Avoiding vendor lock-in appears to be the common wisdom in the IT world, and a lot of people follow it without hesitation. But, increasingly, its validity within the context of modern software development is being questioned.

When people think of lock-in, they usually have the old days in mind. That’s when big companies used to sign multi-million dollar contracts with some big corporation (Microsoft or Oracle for example), and would find themselves completely stuck with that provider for a number of years.

That, of course, was a problem. As the pace of technology increased, you’d find yourself unable to use the latest and greatest because of commitments that your company had with these providers. Moreover, the prospect of changing provider would often appear to be so painful and expensive to discourage everyone from even trying.

But that was a different time.

The question we should ask ourselves is whether that is still the truth in 2019.

These days, vendor lock-in is something that a lot of people are happy to go along with. For example, many people own both a MacBook and an iPhone. That, if you ask me, is pretty locked in! If you’re one of those people, then the more Apple products you use, the more locked in you are.

But the main difference here is that we have a choice. People choose to lock themselves into the Apple ecosystem because they they truly, genuinely like Apple. They love using those products on a daily basis. They feel so much more productive using such products that the thought of moving to a different manufacturer scares them way more than being locked in.

In our normal, everyday life, we go with what works best for us. We adopt solutions from manufacturers/providers that offer us the best experience. We value those products so much that we don’t mind the fact that a potential migration to a product made by a different provider could be somewhat arduous.

2. The best way to benefit from the cloud is to go all-in

By going all-in with the cloud I mean taking advantage of everything that the cloud has to offer without trying to reinvent the wheel. Going half-hearted into the cloud simply doesn’t work. Here’s why.

Lower cost and better performance
Cloud-native tools and products are cheaper and faster. The scale at which a big cloud provider can run something like a database is nearly impossible to achieve on your own. That’s why a cloud provider will always be able to offer such solutions at a much lower price (and with far greater performance) than if you were to run them on your own infrastructure.

Even if you run, let’s say, Postgres on the cloud, it won’t usually be as cost effective as something that was built for the cloud from the ground up (e.g. DynamoDB or Aurora Serverless).

Pace of innovation
AWS releases new services on a weekly basis. At any given time, there may be a recently launched service that will most likely reduce costs, increase performance, and improve maintainability for your application. Adopting these new services can give you an advantage over your competitors and make your life easier.

It’s important that you make it easy for yourself to quickly test and adopt a new service. The key to that agility is to reduce the amount of tools and services that you have to maintain by favouring serverless and managed services over things you have to install and configure yourself.

That’s how you get the most from the cloud, by paying them to do things you no longer want to worry about.

3. AWS is the best cloud provider out there

This is almost uncontroversial. Yes, Google has claimed that they will surpass AWS in the next few years, and Microsoft has been growing at an amazing pace. Nevertheless, if you use these services, you will soon notice how they don’t even compare to AWS.

Google has done some great work, particularly in the area of AI and machine learning. A lot of people use Google just for that bit, and then rely on the other cloud providers for everything else. The most typical comment I hear from people who have tried Google Cloud after using AWS or Azure, is ‘they have some good stuff, but overall it looks like they’re not even trying!’. It’s truly amazing when you think of just how massive Google is. When you compare their cloud offering with Amazon’s, it’s tiny.

Microsoft is a different ball game. Here is a company that has been innovating (and growing) a lot. With the purchase of LinkedIn, GitHub, and with strong cloud products such as Office 365, they have a very strong offering. Microsoft Azure can be a valid option for some companies. However, AWS still offers the largest depth of services with 140 across many different categories. So if you go down the Azure route, the possibility of having to rely on AWS for some things is a real one.

Conclusion

So-called vendor lock-in in the modern era is nothing like it used to be. If you feel that AWS is the right cloud provider for you (and we’ve established that there are very good reasons for thinking that), then you should embrace it fully. By doing so, you will leverage the amazing cost savings and increased performance that the cloud has to offer.

Resources

Image credit: Photo by Dallas Reedy on Unsplash

Are you using AWS for your projects? What’s your experience with it? Leave your impressions in the comments below.

How I plan my week

Tue, 26 Mar 2019 08:07:41 +0000

Planning my week in advance is a great way to get clear on my priorities and make sure the most important things are done before anything else.

There are many ways to plan your week, but here I’m going to list out what works for me.

Long term vs short term planning

In the past, I would focus on either longer term planning (using Michael Hyatt’s Life Plan), or simply daily planning. Now, those are both extremely valuable ways of planning your time, and they are by no means unhelpful.

But the issue with focussing on the long term is of course that you can’t be very specific. You can only have these grandiose plans for the future, but it’s not really going to help you with what you need to get done in the immediate term. Don’t get me wrong, it’s helpful to know where you’re going in life. But it’s also really hard because nobody knows the future.

For example, in 2015 I came up with my “Life Plan” for the next 3 years. It was full of really nice goals in all areas of my life. In some areas (e.g. spiritual and relationships) I was able to keep it fairly high level so that by 2018 those goals were still somewhat relevant. In other areas, particularly on work, it just didn’t apply anymore. The things I thought would help me advance with my career in 2015 simply were no longer relevant in 2018.

This is, again, not to discard or underestimate the value of long term planning. It is simply to say that it is very hard to get it right. Moreover, I’ve been increasingly intrigued recently with the “no-goals” philosophy. Jason Fried and David Heinemeier Hansson talk about this extensively in their recent, fascinating book on how they’ve built a so-called “calm company”.

I’m still not entirely sure about the value of having absolutely zero long term goals, but the idea that I should simply keep doing what I’m good at (and keep getting better at it), while leaving the rest to take care of itself is something that resonates with me.

The other thing that I’ve tried is super detailed daily planning. That is, planning each and every minute of my day in advance and then making sure I stick to the plan. I did it for a while, and it helped me. I got inspired by Cal Newport’s method. I would recommend that anyone do it for at least a season, as it will really help getting clarity on how to best organise your day.

For example, some of us get a lot more done in the morning, so jelously blocking a few hours from, say, 9:30 to 11:30, to close yourself in a meeting room and get some deep work done could be the best productivity hack you’ve ever implemented.

My current weekly planning methodology

While I’m currently not adopting any specific framework for long term planning, and I’m no longer planning every single minute of my day, I have come to appreciate the value of planning my week in advance.

My method is simple.

On Sunday evening (or Monday morning) I spend some time reviewing the past week, getting current on the things that I have to get done on the upcoming week. That also includes reviewing meeting notes and/or emails from the past few days, just to make sure I don’t miss out on any pending task.

Then I make a list.

I love the idea of the Big Three. That’s basically saying that you need to choose the 3 big items that you want to accomplish in the following week, and just get intense on those ones. That has many benefits, such as that you don’t get overwhelmed by too many tasks, but also that you can focus on quality rather than quantity. You do less but you do it really well.

Unfortunately, that doesn’t really work for me as I usually have more than three things to do within a week. I try to keep my list of things to do between 5 and 7 items.

It’s important to notice that at this stage, I’m not going too specific. The items that I’m listing are really projects, or portions of projects, rather than specific tasks.

How I choose what to do every day

Once I’m clear on what’s most important for the week, I spend a few minutes every morning reviewing progress on those items and writing done what I can do today to move the ball forward.

This is where I get specific. Whatever it is that I’m writing down, it needs to be achievable within the day. It can’t be too high level.

I find that it’s usually a good idea to undershoot here. It’s a lot better to have a lot of small things done at the end of a day than a couple of not-quite-finished tasks that I will need to drag into the following day.

Digital vs analogue

Working in technology, it is only natural that when I was looking at ways to plan and track my work, I first went and looked for the best app to help me do that.

But about a couple of years ago, I came across this book, and it has affected my thinking so much that I’m now a lot more inclined to use analogue solutions for crucial planning such as this. Being able to detach from my laptop or phone, alone with my notebook, and focus on what I need to get done is invaluable.

Of course, there are no hard and fast rules here. While the benefits of using analogue tools for planning your life are not a mystery, there are plenty of great productivity apps. Before I went analogue, I was keeping a simple document for the current week on Dropbox Paper. It worked reasonably well.

How do you plan your week?

I hope this will give you some inspiration or ideas for how to plan your week and days more effectively. I’d love to hear what you’re using, and what is currently working for you.

Image credit: Unsplash

What’s your experience planning and organising your week? Share your thoughts in the comments below.

Why unnecessary variables are bad for your code

Tue, 18 Jul 2017 08:07:41 +0000

Variables are one of the most basic elements of programming. They’re usually one of the first things you learn about in programming courses, so your habits for using them form early.

It’s no surprise that every so often we see them being misused and even abused. They can be powerful if used properly, but they can also point to a lack of proper design when not used in the correct way.

It’s a particularly bad practice when variables are used to explain what is going on in the code. It’s no different than the issue of comments. The code should be self-explanatory, and neither comments nor variables should be used as shortcuts to achieve that.

When reviewing someone else’s code, we should be looking at the variables that are being created and assigned all over and ask ourselves whether there could be a more efficient, cleaner way to perform the same operation.

The myth of increased readability

Popular programmer wisdom seems to favour creating variables for the sake of readability. But this has two main issues:

1. Variables actually make readability worse.

As explained by Yegor Bugayenko, more variables in the code mean more lines of code and more values/names that need to be kept in mind while scrolling through the code. He argues that it is much easier to read a line of code that contains all you need to know, rather than having to constantly double-check what the content of a variable is before you can understand the purpose of the line of code you have in front of you.

2. A variable for readability is almost invariably a shortcut.

If the main purpose of a variable is to increase readability, then we know that a shortcut has been taken. Readability comes for the most part from good design. Using a variable with the intention of making the code more readable comes very often as an afterthought. And an afterthought is hardly ever synonymous with carefully thought-out design.

Variable misuse scenarios

It’s important to know what variable misuse scenarios look like so that we are able to identify them while reviewing someone else’s code. It also makes sense to know some practical ways to “fix” the issue and why it’s necessary.

A small variable

When a small variable is used for the sole purpose of increasing readability, it looks something like this:

var Author = Book.AuthorName();
// ...more code here...
console.log("The author of the book is: " + Author);

This helps no one. We’re simply adding yet one more line to the codebase. While at a superficial level it might look as if this is helping readability, it’s offering no real, substantial help over the use of a method from the object Book, namely Book.AuthorName(). In fact, it’s making things worse. Whoever reads this code is always going to need to look for where the variable Author is being defined to identify its content.

The above example can be improved by changing it to look like this:

// ...more code here...
console.log("The author of the book is: " + Book.AuthorName());

Of course, this is a rather basic and oversimplified example. But by understanding the principle, you should be able to recognize similar, more complex cases of abuse of a small variable.

A long variable

There is a place where it might seem harder to argue that a variable for the sake of readability is unjustified. That’s the case of a variable containing a much longer value.

Imagine a value that looks like this:


html = HTML(open(url))
author_name = html.css('a')[5].text.chomp
puts "The author of the book is #{author_name}."

Now, it would look horrible to concatenate all of html.css('a')[5].text.chomp directly with the printed message, right? Right. But the solution is not to put that value into a variable. Rather, it should all be contained within a small function or object.

This is what it could look like:

def author_name_from_html(html)
  return html.css('a')[5].text.chomp
end

And then what gets printed is:

puts "The author of the book is #{author_name_from_html(html)}."

Here are the reasons why this is a much better solution:

It’s cleaner. There are no variables all over the place, just a small, handy function.
It’s reusable. You can use that function everywhere in the system. That’s not the case with variables, unless you adopt global variables, which is usually not a good practice.
It’s easier to understand. You don’t need to look at the implementation details of the function to know what’s happening. By quickly glancing at its input (html) and its output (a string), you know exactly what to expect whenever you’re met with that function.
It’s testable. This should actually be the first point! You can write a test over that function and make sure that it does exactly what it’s supposed to. No such luck with a variable.

Nothing replaces good design

At this point, it should be obvious that variables are not as innocuous as some might think. Being a rather easy tool to use, they can very often tempt the developer to use them as a shortcut to make the code look better. But as we’ve seen, they are not the best way to improve hard-to-read code. In fact, they often make matters worse.

Nothing can replace a good design. When doing a code review, take the time to carefully consider whether the variables used actually have a place there. Training yourself to recognize misused variables and to rework that code will make you a much more valuable code reviewer.

Image credit: Flickr

What’s your experience with useless or bad comments? Share your thoughts in the comments below.

This article was first published on TechBeacon.

Basic abstraction techniques: What code reviewers need to know

Thu, 01 Jun 2017 11:46:11 +0000

While having a large amount of code doesn’t necessarily mean the code is complex, there certainly is such a thing as too much code in the wrong place. Programming code can also be unnecessarily complex and hard to follow. As a matter of fact, these two problems often occur together, and it’s something you need to be on the lookout for as a code reviewer.

What’s usually lacking in those parts of code where you find yourself on the verge of throwing in the towel because you just can’t understand what is going on is what we call abstraction. When sections of code get too cumbersome and hard to follow, that’s when you need to employ an abstraction technique. Let’s look at how to spot overly complex code and then simplify it.

The power of abstraction

Two of the main paradigms of computer programming are imperative and declarative. To put it simply, imperative programming means telling the compiler how to do things, line by line. Declarative programming, on the other hand, abstracts away the details of how things are done, in favor of a high-level description of what needs to be done.

The reason declarative programming is deemed by many to be a superior way to write code is that it’s often a lot easier to read someone else’s declarative code than something written imperatively. When you read and work on declarative code, you don’t need to bother with implementation details. You can, instead, focus on the general business logic and only look at the specific implementation of something if you need to.

Too much code?

For code to be easier to read and understand, its business logic should be as obvious as possible. And for that to happen, there needs to be as few “implementation details” floating around as possible.

As an example, look at the following procedural/imperative piece of code:

let userId = "2"
let address = "20 Gortnatra St., Kerrykeel, County Donegal, Ireland"
let supergeocoder = new SuperGeocoder()
let geocoder = supergeocoder({provider: "google-maps"})
let geocoderResult = geocoder.geocode(address)
let gpsCoords = geocoderResult.latitude + ',' + geocoderResult.longitude
let mysql = new MySql();
let connection = mysql.createConnection({
  host : 'localhost',
  user : 'me',
  password : 'secret',
  database : 'my_db'
});
connection.connect();
connection.query('UPDATE users SET address = ?, gps = ? WHERE id = ?', [address, gpsCoords, userId], function (error, results, fields) {
  if (error) throw error;
  // ...
});
connection.end();

It wouldn’t be surprising if it took even an experienced developer several moments to understand what that piece of code is trying to achieve. This is how we could attempt to make it declarative:

let userId = "2"
let address = "20 Gortnatra St., County Donegal, Ireland"
let User = new User(userId)
let Geolocation = new Geolocation()
User.saveAddressCoordinates( Geolocation.coordinatesFromAddress(address) )

The main thing that we’ve done here is remove all the implementation details, such as the database and the geolocation library. We’ve also gotten rid of some database-specific language, such as the SQL query and the database connection data.

Now anyone can read that segment of code and have a pretty clear idea of what its business logic is. The code is clearly trying to save the user’s GPS coordinates by calculating them from a real address.

When reviewing someone else’s code, look for places where implementation details are hindering a clear understanding of the business logic. Suggest practical ways to abstract away the what, leaving only the how exposed. The easiest way to do that is typically by replacing that piece of implementation code with a small function or object, as shown in the above example.

Decompose conditional

Another place where you can find too much code in the wrong place is within if conditions. When reading an if condition it should, again, be immediately obvious to any reader what that condition is trying to accomplish.

Look at the following segment of code:

if (
    (
        Person.age > 65
        && Person.gender == Person.GENDER_FEMALE
        || Person.age > 67
        && Person.gender == Person.GENDER_MALE
    ) &&
    Job.employeeDetails(Person).employedSince >= 1980
) {
    // some code here
}

This huge condition should be replaced by a simple function such as this:

if (isEligibleForRetirement(Person, Job)) {
    // some code here
}

isEligibleForRetirement simply contains all of the conditions we’ve just looked at, but it’s much easier to read, understand, and test. This technique is called decompose conditional.

Once again, what you’re looking for here is a way to understand the business logic clearly and as early as possible while in the process of reading the code.

In the first condition, even after reading it all, you probably still wouldn’t know what that if is trying to do, which is to scan through people who are eligible for retirement. A comment might have been necessary for you to fully understand it.

In the second condition, it’s immediately clear what the business logic is. Unless you care about the specific requirements for a person to be eligible for retirement, you can happily keep reading the rest of the code.

Consolidate conditional expression

Something else to be on the lookout for is when a number of conditions return the same result.

Here’s what it would look like:

if (Job.position == “truck_driver”) {
    if (Applicant.age > 65) {
        return false
    }
    if (!Applicant.hasDrivingLicence()) {
        return false
    }
    if (Applicant.hasCriminalRecord()) {
        return false
    }

    Job.sendApplication(Applicant)
    return true
}

There are a number of reasons why something like this might happen. One of them might be that different developers have been adding those conditions one after the other and have never stopped to think about refactoring the code, sacrificing long-term maintainability in favor of getting the code quickly into production (thereby creating technical debt).

In the example above, being above a certain age, having a driver’s license, and lacking a criminal record are clearly all necessary requirements for the job. The three if conditions can be replaced by a single condition:

if (!Applicant.isEligible()) {
    return false
}

As you’d expect, isEligible() will simply contain all of those conditions. The result is something that is much cleaner and easier to read.

This refactoring operation is called consolidate conditional expression, and it’s a very valuable tool that can be used to make code more concise and easy to read.

Too much code in the wrong place

As you can see, in all of the cases we’ve looked at, the complexity wasn’t due to the code not being correct or necessary. Rather, the main problem was that it was in the wrong place.

Code like that should be abstracted away. It occupies valuable space and slows down the reading of anyone who wants to use and understand that code.

The value of code reusability

Code reusability is at the heart of good software. And for the code to be reused, it’s essential that it be readable and easy to understand.

It’s very important for a code reviewer to keep an eye out for pieces of code that require a great amount of time to be fully understood. Very often, the practical techniques shown in this article will help you abstract away that complexity.

When there is too much code in the wrong place, it becomes much harder to read and understand it. You should always be hesitant to approve code that requires significant effort to be understood. When the business logic is not broadly clear after a first reading of the code, a red flag should be raised, and techniques such as the ones listed in this article should be used to simplify the code and abstract away the complexity.

Image credit: Flickr

Share your abstraction techniques in the comments section below!

This article was first published on TechBeacon.

Useless comments can ruin your code reviews. Here's how to erase them

Mon, 15 May 2017 07:09:32 +0000

When it comes to reviewing someone else's code, how should you approach comments?

Comments are typically seen as a good thing. They can improve readability, offer context as to what the code is trying to do, and help you remember parts of the code that should later be changed or refactored.

But is that all true? Are comments really that good, or can they be the telling sign of a bigger problem?

The hard truth is that, in many cases, comments can point you to inherent problems with the code that you're reviewing. There are also cases when comments are helpful, and you should know how to identify when and where this is the case.

But first, here are some examples of bad comments.

//FIXME

Just look at this piece of code:

$done = false;
$attempt = false;

while (!$done) {
    $attempt++;
    $done = true;

    performSomeAction();

    if (somethingWentWrong()) {
        $done = false;
    }

    // FIXME Should we delay a second or two before retrying?
}

Who wrote that FIXME comment there? Was it the developer who initially built this feature or someone who worked on this piece of code later on?

If you're reviewing this code, you probably have the answer and are probably at the best point in time to stop this from going to the master branch.

There is no reason for that comment to be there. If the question, "Should we delay a second or two before retrying?" is an open one, then the developer should ask the project manager so that an actual answer can be found. If there is no project manager to be found, then, of course, we have a bigger organizational problem. Regardless, the comment here is indicative of something that might be wrong with the project, and that should not be tolerated. This is a classic example of the comment being a symptom of a bigger, in this case organizational, problem.

//TODO

TODO comments are often introduced by the same developer who built the new feature or piece of code that's being reviewed. That's because as she was working on the code, she realized that some improvements could be made, only to decide not to apply those improvements this time around. This could be due to any number of reasons, from time limitations all the way to laziness.

A TODO comment could indicate that some small refactoring may be helpful:

// TODO: do decompose conditional (https://refactoring.com/catalog/decomposeConditional.html) here 
// by replacing this condition with a function
if user.isActive == true && user.Country == "Ireland" {
    doWhatever()
}

Of course, it only takes a few minutes to replace that line of code with something like:

if isActiveUserFromIreland(user) { ... }

There is no reason to clutter your code with a TODO comment when all it does is reminding us of a small, quick refactoring that could help us improve the code. If it’s a small enough change, it should just be done. There is no need to defer that change.

A TODO comment could also point to the need to add something very important, as in this case:

func addUser(name string) {
    // TODO: save user to database
}

Here, this functionality simply doesn't work. That method is meant to save something to the database, and it's obviously not doing it.

While in the first example the change needed was a small one and it should have simply been done, now we're faced with a potentially more complex functionality. Who knows why that's not been done yet? Maybe we haven't chosen a database library yet, or we lack some vital information to proceed. Regardless, that TODO comment is there to clutter our code. It's not the best way to achieve what we want, namely, to remind us that the functionality that saves the user to the database needs to be implemented.

Whether the change indicated by the TODO comment is a small or a big one, it inevitably points us to the lack of a properly used ticketing system such as Jira, GitHub Issues, or YouTrack.

Unnecessary comments

Have you ever heard someone saying that you can never have too many comments in your code? Unfortunately, that's not true. Comments that are not strictly necessary can be instrumental in hiding badly designed code.

Look at this example:

type User struct { ... }
func (u *User) changeName(name string) {
  // verify whether the user has a name already
  hasName := false
  if (u.Name != nil) {
    hasName = true
  }
  
  // if user has a name already
  if (hasName) {
    u.PreviousNames     = append(u.PreviousNames, u.Name)
    u.NameChangesCount  = u.NameChangesCount + 1
  }
  
  // save the new name 
  u.Name = name
  u.FullName = u.Name + " " + u.LastName

  // save initials
  u.Initials = u.Name[0] + u.LastName[0]
}

On top of some obvious problems there, what those comments tell us is that we're dealing with bad design. There are a number of concerns with this function, such as:

Breach of single responsibility principle: The function is doing too much.
Too much procedural code: Those pieces of code could easily be moved into small, expressive functions.

There is more that we could say about this piece of code, but let's focus on the comments and why they're indicative of the above-mentioned issues.

First, if the function had complied with the single responsibility principle, we wouldn’t have needed those comments at all. The function is called updateName, and as long as what’s happening inside the function is the update of a name, then there is no need to add any comment. It’s self-explanatory!

Second, it’s clear in the example that the comments are enabling us to keep adding lines of code doing all sorts of things. A thoughtful programmer would feel unjustified in adding more and more lines of code to a single function. But, because there are nice little comments telling us what’s happening line by line, then it feels like it’s not a big a deal after all.

In the above example, virtually any piece of code that's preceded by a comment should be in its own function. There is no way around that. Comments are only a shortcut in this case, and they should be treated as such.

Commented pieces of code

This is an easy one to identify. A piece of code that used to be running and that gets commented out should have no place in your code. As a code reviewer, it's your responsibility to point this out.

The reasons for leaving a piece of code commented instead of removing it completely are typically related to either not being sure if that code will ever be needed again, or wanting to leave it there as a point of reference for everyone else.

These are not good enough reasons to clutter your code with unused code.

But then, how do you preserve a piece of code that could be needed for the future? The short answer to that is: You don't. The longer answer comes in two points:

You should be working with a modern version-control system such as git. That will always allow you to go back to any prior version of a file and look at the code as it was.
There is no reason for you to believe that the exact piece of code that you're commenting will still be working once it's needed again. You should rather have a place where you can document the way that piece of code used to work on a high-level basis. It should then be implemented afresh if you ever need to in light of the way the rest of the code works now.

What's so bad about comments?

We've looked at a number of specific examples that should help you identify bad comments when you see them as you perform a code review.

But comments should almost always make you stop and carefully consider whether they're needed or not. Here's why:

Clutter. Bad comments add clutter and make the code less readable. It's important to only keep the comments that are strictly necessary.
Bad design. As we've seen in the examples above, comments often tell us that the design could have been way better.
Laziness. Comments can be used as a cheap shortcut to avoid writing proper code according to best practices, as in the "unnecessary comments" example.
No compiler checking. Comments go unchecked by the compiler, which means that there is no way to ever tell us if they're correct or not. Code itself is the most reliable and self-documenting resource.

Good comments

With few exceptions, the only comments that can be considered good are the ones that give us a high-level description of a construct (such as a class, a type, an interface, or a function).

As in this example:

// Car is an interface that can be used to implement Car objects
// It's a generic car and it's not specified whether the gearbox
// is automatic or manual
type Car interface {
  SwitchEngine()
  Gear() Gear
  Colour() string
}

This is a perfectly acceptable comment that gives us context as to what the Car interface is about. We probably would have understood it without the comment as well, which is the whole point. But the comment is helpful and doesn’t take anything away from the correctness of the code.

Code review de-coded

As we've seen, there are some cases (however limited) where comments can be useful for the person reading them. The key is to only add information that the reader could have deduced anyway. Comments should help provide context to a construct, but they should not be used to describe what the code is doing.

It's important to also make sure that comments aren't masking bigger issues, such as fundamental organizational inefficiencies or laziness on the part of the developer who wrote the code. Even though comments are often viewed as positive, it's always a good idea to carefully review code with too many comments mixed in.

This article was first published on TechBeacon.

How to run code reviews in your dev team's workflow

Thu, 20 Apr 2017 08:01:27 +0100

While they’re usually accepted as a good practice, code reviews remain a topic of debate among software engineers. Many programmers still struggle to identify the extent of code reviews’ value and their place within a team’s workflow.

The questions of why you should do them and when have more in common with each other than you might think. To maximize the effect of a code review, doing it at the right time within your development workflow is crucial.

At the same time, trying to artificially fit code reviews within your existing workflow is not always a good idea. If you want to make sure you get the best out of every code review, adding one as just another step to your workflow may not be enough. Some changes to your workflow could be necessary. I’m going to help you answer the simple question:

Before the code gets merged

The most obvious advice when it comes to code reviews is to do them before the new code gets merged into your production or main development branch.

The main reason for this common piece of advice is that you want to be able to find any potential issue before the code makes it to the place where everyone else will be using it.

This does, of course, seem to work better with a distributed version-control system, such as Git or Mercurial, that lets developers work on their own local version of the repository. When the work is completed, they will then request (via a pull request, for example) that someone review the new code. You can set rules so that the code will get merged only once it’s approved by the reviewer.

Note that there is nothing wrong with performing a second code review after the code has been merged, but it would be a mistake for the code not to be reviewed before it gets to that point. Also, post-merge is probably a better time to perform an independent review.

After the tests have been run

Code reviews are no replacement for a reasonably high test coverage. While it’s obvious that a code review itself should help find bugs in the code, your fellow code reviewers shouldn’t be finding regressions—those should be caught by automated tests.

Before a code review, the new code should have sufficient test coverage, and all tests should be passing. If anything breaks, it has to be fixed before someone is asked to review the code.

The reason for this is that you want to give each step of your development workflow its own responsibility. While code reviews can, at times, be instrumental in finding bugs, they are not the primary instrument for that purpose. This work needs to be done by a robust collection of unit, functional and integration tests.

There’s always the possibility of a bug not being caught by any test, and that’s where code reviews typically become critical. Another possibility is that the tests are badly written and so, while they pass, they don’t really make sure that the code’s business logic is doing what it’s meant to do. Again, this is where a good code reviewer will step in.

As far as the process goes, the priority is to make sure that all tests are automated and run explicitly when new code gets pushed. The code reviewer should be able to fully trust that the committer of this new piece of code has run the tests and that the code compiles.

The results of the automated tests should also be clearly visible to everyone. Your automated tests should be integrated as a required step before merging, with the results of those tests publicly available. This is very easily done on GitHub and similar platforms.

The case for small branches

I already mentioned how a bad development workflow might negatively impact the effectiveness of code reviews. A bad practice that many teams seem to fall victim to is creating big tickets/tasks (and therefore branches) that someone will be working on for several days, weeks, or even months.

You want to do the opposite. Keep your tasks (and branches) as small as possible. This will make writing each task’s code much easier for developers, and it will also be a more manageable review for the code reviewer.

Reviewing a day’s worth of code is a task that can be done in a few minutes, and the review will likely be much more accurate than if the reviewer had to check several weeks’ worth of work. The chances of missing something crucial when reviewing big chunks of work, rather than small ones, go up exponentially.

Short reviews, often

Reviewing other people’s code shouldn’t take hours out of an engineer’s day. By keeping tasks small, you allow the senior engineers or software architect to do code reviews often and for short sessions.

Gareth Wilson, in his article on effective code reviews, outlines how running code reviews often and for short sessions has a couple of benefits.

First, it doesn’t interrupt your flow. Engineers know how important this is; there is nothing worse than getting interrupted or having to do context switching while you’re “in the zone.” As Cal Newport, a computer science professor explains in his book Deep Work, in order to produce a high-quality piece of work, we need prolonged, interruption-free times of deep thinking and undivided focus. Short code reviews take just a few minutes, allowing engineers to have a short block of time to focus on them and larger blocks of time to focus on writing code.

Second, it’s less frustrating for the person who wrote the code, since he won’t need to wait several days for someone to finish reviewing his work. Instead, he can get quick feedback and apply the requested changes while everything is still fresh in his mind.

Are “pair reviews” a good idea?

Very often in development teams, the code’s committer will sit together with the reviewer and look at the new piece of code. The reviewer will scroll through the new code, while the committer will try to explain what’s going on.

This may sound like a good idea, and it definitely has its place in various instances, but it’s not the most effective way to run a code review, because it misses an important reason for doing the review in the first place: Namely, the review should verify that the code is easy to understand and self-explanatory.

The only way to make sure this is the case is for the reviewer to look at the code alone and try to get an understanding of it without asking the code writer any questions. If the reviewer is unable to understand the code, that is typically a sign of badly written code or a lack of good unit tests.

Should the architect or senior engineers do the review?

Ideally, both another engineer and the software architect should perform code reviews.

A code review performed by a peer engineer has a different purpose than one performed by the architect. While both of them will be looking for a lot of the same things, such as bugs, lack of tests, readability, and so on, the architect will also use the code review to make sure the team is following architectural guidelines.

As Yegor Bugayenko explains, code reviews enable the software architect to enforce the design and architectural principles of the project.

Another engineer can always identify a deviation from the guidelines and best practices followed by the team, but it’s the software architect, says Bugayenko, who needs to use code reviews to prevent his vision from being destroyed.

In other words, code reviews are a crucial tool for a software architect to make sure that the team is following the architectural direction that’s been established.

Tools matter

A better tool will allow you to perform better code reviews. Period. It’s as simple as that. Downloading the new code, screening it file by file, and then writing your comments somewhere else is not only an inefficient use of time, but also a great way to ensure that your code review will not be very effective.

Modern code review tools such as GitHub, UpSource, and Crucible allow you to perform code reviews quickly, hold conversations about a piece of code, and, most importantly, easily verify that the suggested changes have been made.

Companies should invest in code review tools because they are a relatively cheap and simple way to make sure that the new code doesn’t cause any damage once it gets introduced. Making sure that your team has the best code review tools is a great way to protect the quality of your software and safeguard your customers from potential new bugs.

A simple but powerful tool

Code reviews are one of the simplest tools we have to ensure the highest quality of our software. But their effectiveness will greatly increase or diminish based on how well code reviews fit within the development workflow and on whether we’re willing to invest in them.

Changing or adapting some aspects of your workflow based on the advice I’ve given will help your team perform more effective code reviews.

How does your team run code reviews? Share your best (or worst!) practices in the comments below.

This article was first published on TechBeacon.

8 best practices for microservices security

Fri, 17 Feb 2017 08:20:55 +0000

There is virtually no situation in software architecture that entirely frees you from security considerations. With microservices, some issues become more distinct and a lot harder. However, there are also a few features of microservices that can bolster security.

With microservices, the network is still a bottleneck. Things like access control, which the industry already understands thoroughly within the realm of monolithic applications, assumes a new, almost unexpected, level of complexity. This paves the way for debates and scrutiny over whether a microservices architecture actually solves more problems than it creates. Your decision to use microservices should always be conditional.

When you’ve done your due diligence and decided that microservices are right for you, it’s time to make sure that all of your applications’ security demands are met. Here are eight best practices for securing your microservices.

1. Use OAuth for user identity and access control

The overwhelming majority of applications are going to need to perform some level of access control and authorization handling. What you want to avoid here is reinventing the wheel. OAuth/OAuth2 is practically the industry standard as far as user authorization goes. While building your own custom authorization protocol is clearly an option, many out there don’t recommend it unless you have strong and very specific reasons for doing so.

While OAuth2 isn’t perfect, it’s a widely adopted standard. The advantage of using it is that you can rely on libraries and platforms that will greatly accelerate your development phase. By the same token, several solutions for improving the security level of your OAuth-based authorization service have already been built by some of the biggest companies and smartest engineers around.

2. Use ‘defence in depth’ to prioritize key services

Assuming that a firewall on your network perimeter is enough to protect your software is a big mistake. “Defense in depth” is defined as “an information assurance concept in which multiple layers of security controls (defense) are placed throughout an information technology system.”

In plain English, what you need to do is identify what your most sensitive services are, and apply a number of different layers of security to them, so that a potential attacker who is able to exploit one of your security layers will still have to figure out a way to beat all your other defenses on your critical services. This is by all accounts easier said than done, but several resources are available.

Security is typically a job better left to experts and not to amateurs. A proper defense in depth strategy is more likely to succeed if it’s established by people who actually know what they’re doing.

What’s great about microservices is that they make it easier to adopt this strategy in a very granular and strategic way—by focusing your security efforts and resources on specific microservices. The architecture also makes it easier for you to diversify the layers of security you wish to adopt on each microservice. By so doing, an attacker who is able to exploit one of your services may not necessarily be able to figure out how to exploit the second one.

3. Don’t write your own crypto code

Over the years, many people have invested incredible amounts of money, time, and resources into building libraries that handle encryption and decryption. If you hired 10 smart and competent security people, put them all in a room and asked them to come up with the best possible library for encryption and decryption, I doubt they would come up with something as good as the best open source crypto libraries that are already out there.

Most of the time, when it comes to security you shouldn’t try to roll your own new solutions and algorithms unless you’ve got strong and specific reasons to, and you’ve got people skilled enough to create something nearly as good as the open source tools already available (tools that have been heavily battle tested by the community).

In most cases, you should use NaCl/libsodium for encryption. It’s been around for several and it’s fast, easy to use, and secure. While the original implementation of NaCl is written in C, it also supports C++ and Python. And thanks to the libsodium fork, several adapters for other languages like PHP, Javascript, and Go are available.

This section wouldn’t be complete without mentioning the wildly popular Bouncy Castle library. If you’re working with Java or C#, your best bet is to go with this one. If you want to learn more about encryption, read this developer’s guide.

4. Use automatic security updates

If you want your microservices architecture to be secure and scalable at the same time, it’s a good idea—in the early development phase—to figure out a way to automate or at least keep all of your software updates under control.

High testing coverage here is more essential than ever. Every time a part of your system is updated, you want to make sure you catch any issue early enough and in as much detail as possible.

Make sure that your platform is mostly “atomic”. What that means is that everything should be wrapped within containers so that testing your application with an updated library or language version is just a matter of wrapping a different container around it. Should the operation fail, reversing everything is fairly easy and, most importantly, can be automated.

CoreOS, RedHat’s Atomic Linux, and Ubuntu’s Snappy Core are also projects you want to keep an eye on, as they try to bring about the same concept on an OS level.

5. Use a distributed firewall with centralized control

For the most part, this is still uncharted territory, but I believe that a firewall that allows users more granular control over each and every microservice (as attempted by Project Calico) has got to be the way we build firewalls for microservices. If not now, at least at some point in the future.

6. Get your containers out of the public network

Amazon, with their AWS API gateway, probably made this whole notion more mainstream and easy to adopt than anyone else before.

An API gateway establishes a single entry point for all requests coming from all clients. It subsequently knows how to provide an interface for all of your microservices.

By using this technique you can secure all of your microservices behind a firewall, allowing the API gateway to handle external requests and then talk to the microservices behind the firewall.

Moreover, as the Netflix experience teaches us, using an API gateway is a great way to optimize requests based on the client, especially in the case of mobile devices.

7. Use security scanners for your containers

Within your automated testing suite, it would make sense to include periodic vulnerability and security scanning for your containers. The chief open source actor in this space appears to be Clair, from CoreOS. Docker Security Scanning and Twistlock are a couple of commercial options.

Something else to keep in mind here is that the container image itself may not necessarily be trusted unless its signature has been verified. rkt does that by default, while Docker introduced a similar feature a while ago after several security vulnerabilities were found.

8. Monitor everything with a tool

You can’t afford to run a distributed system without a solid, advanced, and reliable monitoring platform. Several solutions are available out there, but the one that was built specifically with microservices in mind and has been around the block is Prometheus.

Built originally by engineers at SoundCloud, Prometheus is an open source monitoring platform and a part of the Cloud Native Computing Foundation. It’s being supported and adopted by some of the biggest players in the industry, like SoundCloud themselves, CoreOS, and Digital Ocean.

Other monitoring solutions include InfluxDB, statsd and several well-known commercial platforms.

Don’t reinvent the wheel

While the above is not intended to be an exhaustive list, it touches on the issues you are most likely to face when building applications based on a microservices architecture.

When it comes to security, reinventing the wheel is rarely a good idea. Always be researching the best practices adopted by the industry and suggested by experts.

What are the best practices or resources on which you rely for securing microservices? Share in the comments below.

Here are some additional resources I’ve bookmarked:

Project Cockpit - administer your Linux servers via a web browser
Securing Micro-services with a Distributed Firewall (video)
Docker and High Security Microservices
Defense in depth conference
Network Perimeter Security: Building Defense In-Depth (book)
Docker Containers: Build and Deploy with Kubernetes, Flannel, Cockpit, and Atomic (Negus Live Linux) (book)
Container Defense in Depth
API gateway pattern

This article was first published on TechBeacon.

Serverless: What it is and when to use it

Mon, 06 Feb 2017 16:53:45 +0000

“Serverless” is a term you may have heard of recently. It’s also known as Function as a Service (FaaS), and it basically involves storing your functions somewhere in the cloud, and then invoking and running them via a trigger. But how does a Serverless system actually work? And is it something we should be looking at?

How it’s presented

The concept of Serverless has been made popular by AWS with its relatively new product Lambda. It’s pitched as a way to “Run code without thinking about servers.” Another important element of it, which AWS works hard to make us notice, is the notion of paying only for “the compute time you consume”.

Interestingly, Google Cloud has a similar (if not identical) product called Cloud Functions, which is presented from a different angle: “A serverless platform for building event-based microservices”. A more architectural perspective, if you will.

What it essentially is

If your PaaS can efficiently start instances in 20ms that run for half a second, then call it serverless. https://t.co/S3YzvqFYLR
— adrian cockcroft (@adrianco) May 28, 2016

When it comes down to it, you could look at Serverless as Heroku on steroids.

Clearly, the servers are still there. It’s just that those companies (AWS, GCE, etc) have gotten to the point where they can get their servers up and down in a matter of milliseconds. So, as they’d ask you, if you have a functionality which is only running several times a day on an on-demand basis, do you really need a server dedicated to it and running all day long?

Looking at it from a microservices perspective, that’s what we would’ve done. We would have split that critical functionality into a separate microservice, running on its own server. Then, we would have made sure we could expand that server’s capabilities in as painless a way as possible.

Serverless promises to take care of some of that for us. Firstly, it’s supposed to save us some money by only making us pay for the computing resources we need. Then, it takes care of scaling the servers up and down based on the need of the moment.

On a good old PaaS platform such as Heroku, we would have had the same peace of mind of not having to think too much about servers setup and provisioning. But costs could have gotten prohibitively expensive once we reached a certain amount of computing operations. And, of course, we would have had to take care of scaling up or down ourselves.

Do we need it?

Looking at Serverless as a “PaaS on steroids”, whilst clearly an over-simplification, can be helpful. Not only it helps us understand Serverless better, it also gives us some direction as to when and whether we need to consider Serverless for our software.

In most cases, if you had good reasons for not using a PaaS platform, then those same reasons probably still apply for Serverless.

If you’re a happy PaaS user, then Serverless may be an option to consider, but it won’t always work. Similarly, if you’re doing microservices, Serverless is again something you can keep an open mind about.

So, while Serverless and traditional PaaS are not exactly the same thing, they can be seen as solving similar problems though they have different use cases.

When to use it

The most typical and realistic scenario for Serverless is the following:

You have a system built with a microservices architecture. You identify a specific operation within your system which takes considerable computing effort, with periodical and unpredictable spikes in traffic. Moreover, this operation is only activated when triggered by something else. In other words, it’s not always on. That’s a perfect place to try to port that particular operation into a Serverless function.

I mentioned having a microservices architecture. That doesn’t necessarily have to be the case, but it seems quite obvious to me that if you’re doing microservices then your system is probably more prone to further decoupling and separation of a component from the rest of the system.

If your system is a monolith, then you probably have your good reasons why you want to keep everything within the same codebase. Nevertheless, Serverless might still make sense to you. You could, again, identify the single most critical operation within your monolith and turn it into a Serverless function; from there, even a gradual migration to a microservices architecture might just feel more natural.

When NOT to use it

If you’re not already on the cloud, then Serverless is really not a good idea. Services like Lambda work well when tightly integrated with the AWS platform. For example, you trigger an event when storing a file on S3, then you run your Serverless function which does something to that file, then you store the changed file on S3 and perhaps add/update a record on DynamoDB/RDS to indicate that the operation was successful.

If you want to use Lambda but you’re not already on AWS, then Serverless is just not for you. The same applies with GCE’s Cloud Functions and Azure’s Functions.

In other words, Serverless is one of the best ways to lock your system to a specific vendor. If that doesn’t sound like a great idea to you, then stay away from Serverless.

Conclusion

To conclude, Serverless/FaaS is an advancement in the way we operate our servers, and we should all be excited about it. It can be a sensible choice if you have no issue locking yourself to a particular vendor, and if your operation is at the same time:

small
critical
intense computing-wise
not always on

In any situation where all of the above conditions don’t check out, a more “traditional” approach is recommended.

Have you used a Serverless platform? Do you agree with my definition and guidelines? Feel free to leave a comment below.

More resources:

Serverless (Martin Fowler’s blog)
Building Serverless Architectures (Book)
AWS Lambda: A Guide to Serverless Microservices (Book)
AWS Lambda in Action: Event-driven serverless applications (Book)
Serverless framework

My tech predictions for 2017

Tue, 03 Jan 2017 01:32:26 +0000

We’re now a few days into the new year, and it’s probably a good time to look into what might or might not happen during the next 12 months.

It’s a fun exercise, and of course I might be wrong about some or all of the following predictions. They’re based on my personal observations of what I see happening in the industry at the moment.

#1: Apple

Apple is going to keep trying hard but it is unlikely to release anything truly useful and game-changing over the next year.

As with the new MacBook Pro (the one with the touch bar), they’re going to keep releasing products that are only marginally better from a purely technological point of view, with nice and shiny features (like the abovementioned touch bar) that are driven by marketing and not an overall technological/product vision.

Moreover, a sense of discontent seems to be around, with people complaining and even returning their brand new MacBook Pros.

Also keep an eye on: While 2017 might be too early, as an old Linux enthusiast I really hope someone will eventually come up with a Linux-based solution that can stand against MacBooks both in terms of hardware and software.

#2: Docker

More tools are going to come up with the goal of making Docker easier to use. In particular, Docker Compose is going to become production-ready, and keeping Docker commands into Compose’s easy-to-read yaml files is going to become most developers’ preferred way of running Docker apps, as opposed to having to remember huge, unreadable command line commands.

Also keep an eye on: CoreOS’s rkt as a viable (and more secure) alternative to Docker.

#3: Kubernetes and Openshift

Kubernetes is going to become the de-facto industry standard for container orchestration. At the same time, solutions like RedHat’s OpenShift are going to make it easier than ever to benefit from the enourmous power of Kubernetes. Currently, Kubernetes is already seen as the most complete solution, but relatively hard to setup and work with. With OpenShift reaching its more mature stage, small and big companies are going to look at it as a way to ease themselves into the world of Kubernetes and container orchestration.

Also keep an eye on: Weave.Cloud as another possible Kubernetes-based alternative to OpenShift. Weave offers an impressive set of tools that can be very helpful when building software with a microservices architecture.

#4: Microservices

As we move away from the hype of microservices being the solution to all of humanity’s problems, there are going to be more people talking about when it’s not a good idea to build software with a microservices architecture. At the same time, tools that help us manage a distributed architecture are going to reach a higher level of maturity, making it easier than ever before to work with microservices.

#5: AI

There’s going to be more clarity and (hopefully) less plugging around the topic of Artificial Intelligence. As we realise that completely replacing humans may never be possible any time soon (if ever), more resources are going to be invested into smaller, practical projects and technologies that use the power of machines to actually make us more productive and improve our lives in meaningful ways.

This is the sensible way to look at this topic. The purpose of software is to help us live better lives and achieve more and more efficiently. To think of software and, by extension, artificial intelligence only as ways of replacing humans isn’t helpful and isn’t going to get us anywhere.

I hope you enjoyed reading my personal predictions for 2017! I’d love to hear what your thoughts are and what you think we’ll see more (or less) of over the next year.

Is object oriented programming dead? Not by a long shot

Tue, 22 Nov 2016 08:16:23 +0000

Is object-oriented programming (OOP) any good, really? If not, is it just plain bad, or are we simply not doing it right? Are OOP languages even in your future?

You’ve learned about OOP, and you’ve probably done it, or at least you think you have. You listened to other people tell you that it’s the right way—or the wrong way—to do things. You’ve spent long, excruciating hours listening to your computer science teacher talking about how “Toyota” inherits from “car,” and so on.

In real life, though, you may not have found it useful. Perhaps you’ve worked with OOP languages, and at times it seemed more like a restriction than something that actually helps. Perhaps it has bothered you so much that you’ve welcomed the arrival of different and more modern paradigms.

You probably identify with at least some of the above sentences. So you may be asking yourself:

Why OOP?

I enjoy writing code in an object-oriented fashion. Thinking in terms of objects gives me a model that, when followed consistently, produces code that is well organized, and easy to understand, test, and refactor.

Some people argue that OOP doesn’t really work, or that you don’t need it to build modern software. But while I don’t think OOP is the ultimate paradigm, it is helpful. A bunch of procedural code, however nicely organized, is simply not the answer.

But what about functional programming, you might say? That’s all fine and good. Many people say OOP and functional programming don’t necessarily conflict with each other. In fact, they complement each other.

Understanding OOP, polymorphism and immutability

Robert Martin has argued that the biggest benefit you can take from OOP is polymorphism, the notion that you can define a single interface with multiple underlying implementations. As software engineers, we wouldn’t want to lose that. One of polymorphism’s many benefits is that lets you attain inversion of dependency. In OOP, this means that “both high- and low-level objects must depend on the same abstraction,” according to Wikipedia. In practice, this translates into software components that are no longer highly coupled. Each component, both high-level and low-level ones, can be easily replaced.

Martin further states that the biggest benefit you can take from functional programming is immutability, the idea that we design functions or objects so that they always return the same result, given the same values as parameters. Much has been said about this topic, and it’s apparent that its many advantages, such as code maintainability and lack of side effects, far outweigh possible downsides, such as a lack of flexibility of sort.

It shouldn’t come as a surprise, then, that immutability has been a recurring theme among people who have done OOP for a while. In his book, Effective Java, Joshua Bloc says that “Classes should be immutable unless there’s a very good reason to make them mutable… If a class cannot be made immutable, limit its mutability as much as possible.”

Yegor Bugayenko, CTO of teamed.io, is one of the strongest advocates of a return to pure OOP. He writes extensively about the advantages of immutability, saying that “…all classes should be immutable in a perfect object-oriented world.”

There is a lot more to it, and Yegor himself recently wrote a clarifying article, “Gradients of immutability,” where he explains how an immutable object doesn’t always need to have a rigid structure.

This is a big topic, worthy of further study. But by now it should be clear that one good reason to do OOP is so you can retain polymorphism. Not only is that possible, but many people recommended it. While we do OOP, we should also hold on to as much immutability as possible, thereby bringing the single biggest advantage of functional programming into OOP.

If we could get back to the primary advantages of OOP, and get rid of all of the unnecessary ideas that have cluttered it to the point where most programmers have felt like OOP had become counterproductive, we could add more value to the way we write software.

I’ll get back to what this type of stripped-down, or pure OOP, looks like in a minute. But the fact is that real OOP may look quite a bit different from the way you have understood it up to now.

At this point, it’s worth asking the question: why objects? Why do we need to design programs as a number of objects, as opposed to functions, or procedures?

Use objects as a way to understand the world

David West, the author of “Object Thinking,” said in a recent interview that “You look at the world around you and you don’t see functions, but you see objects.” He’s on solid ground here: Philosophers have used terms like abstract and concrete objects to describe the world around them for a long time.

Objects begin to make sense once we stop thinking in terms of bits and bytes, or lines of code. Thinking in terms of objects also lets us stop seeing ourselves as simply people who write code. Our job is really about solving problems, and finding meaningful ways to represent the domain in which we operate. It’s a completely different approach from the one that most people take when writing software. It’s the notion that “programs must be written for people to read, and only incidentally for machines to execute,” as Harold Abelson, Gerald Jay Sussman and Julie Sussman state in their book, Structure and Interpretation of Computer Programs.

Thinking in an object-orientated way is a great way to understand the problem you are trying to solve. It allows you to build software that is more maintainable, and easily understandable, for other people.

By contrast, when you write code in a procedural way, you’re really putting yourself in the computer’s shoes, so to speak, and writing lines of code in the way it will eventually receive your commands anyway. That’s convenient, if you’re writing a quick 10-lines-long script, but less so if you’re building a complex system on which many programmers must work. Eventually, common sense says you should somehow organize this code into smaller chunks, each containing a different set of procedures (or functions) and representing a distinct domain of sort. In the end, hardly any of this will feel natural, or even logical.

In this case, had the programmer originally designed the software in an object-oriented fashion, instead of having a lump of disorganized code, the software would be well structured and easy to understand. This isn’t necessarily a bad thing for the computer, as often code that’s well written and easy to understand is fast to execute, with less chance of memory leaks.

The source of all that OOP criticism

The problem with a lot of the OOP criticism out there is that it tends to refer to things that in popular programmer culture have become associated with OOP, but shouldn’t be.

One example is object-relational mapping (ORMs). Truthfully, you could drop the “O” in this acronym, as there is no way to directly represent a relational database table in a real-world object. Then there’s accessors and mutators (or getters and setters) as they will inevitably change the face of your object into a big “data bag,” throwing encapsulation and data abstraction out the window. The absence of interfaces is one of the main causes for tight coupling (or lack of flexibility) in our code. Finally, static methods turn what may have started as an object into a big collection of functions, or procedures, which was never OOP’s purpose.

Quite a few practices have somehow become part of our daily work but were never supposed to be found in OOP. In fact, using them takes us very far from truly object-oriented code. As a result, we lose out on many of the advantages we could have experienced.

If you were to take a codebase and eliminate at least two of the above mentioned practices, the face of your code would experience a noticeable change for the better.

OOP done well comes with a lot of benefits. But it’s absolutely necessary to remind ourselves what it is that OOP really is, and free ourselves from the many bad practices that have nothing to do with true object thinking.

This article was first published on TechBeacon.

Talks

Mon, 17 Oct 2016 00:00:00 +0000

This is a list of talks I’ve given so far.

If you would like me to speak at your event/meetup/conference, feel free to contact me.

2024

AWS Cloud Day Dublin
Dublin (Ireland) 🇮🇪
10 October 2024

AWS Community Day Belfast
Belfast (United Kingdom) 🇬🇧
6 September 2024

ServerlessDays Belfast
Belfast (United Kingdom) 🇬🇧
23 May 2024

2023

4 Strategies to Build Incredibly Performant Serverless Apps [slides]
Serverless Architecture Conference
Berlin (Germany) 🇩🇪
22 October 2023

2022

The 1 Priority Change You Need to Make to Test Your Serverless App Serverless Summit
Remote 🌐
16 November 2022

Serverless for Startups [slides]
Serverless Architecture Conference
Berlin (Germany) 🇩🇪
18 October 2022

2018

I’ve been to the land of serverless and I’ve come back to tell you all about it [slides] [video]
Dublin Microservices Meetup
Dublin (Ireland) 🇮🇪
29th November 2018

2017

Serverless: what it is, when to use it
Waterford Tech Meetup
Waterford (Ireland) 🇮🇪
29th November 2017

Microservices won’t improve your code quality [slides]
Agile Eastern Europe Conference
Kiev (Ukraine) 🇺🇦
7th-8th April 2017

OOP is not dead [slides]
PHP London
London (UK) 🇬🇧
2nd March 2017

2016

Mistakes made and Lessons learned with PHP (Panel) - PHP Dublin, Dublin (Ireland) 🇮🇪; 4th October 2016

Healthy OOP in Javascript - BelfastJS, Belfast (UK) 🇬🇧; 31st August 2016 [slides]

Microservices won’t improve your code quality - PHP Dublin, Dublin (Ireland) 🇮🇪; 11th August 2016 [slides]

When to use microservices - Video interview, Dublin (Ireland) 🇮🇪; 11th August 2016 [video]

Technical Debt is not a unicorn - Corkdev.io, Cork (Ireland) 🇮🇪; 19th July 2016 [slides] [video]

When to use microservices (video)

Fri, 09 Sep 2016 08:05:54 +0100

A few weeks ago, after giving a talk at PHP Dublin, I was interviewed by the fine folks from the UXDX conference.

The topic of the interview was “When to use microservices”. It was partly inspired by the talk I gave that very night. In the interview, I do my best to define what microservices are and when should companies adopt them. I then proceed to give insights as to how to succeed from both a code quality and a project management point of view.

UXDX is a conference that will run in Dublin, Ireland, on November the 2nd, 2016. The event’s goal is to focus on improving the team behind the product. There will be a set of great speakers from some of the best companies in Ireland and not only. If you can go, you shouldn’t miss it. You can get your tickets here.

This is the video of the inteview. It’s less than 7 minutes long, but I think it contains some interesting points which you might want to look into.

Direct link: https://youtu.be/MxdynUAGQGc

Have questions or comments regarding what I just said in this video? Feel free to message me!