James Lewis says that the concept of Microservices has become a guiding principle in modern software development. How can organisations take advantage of this and what approach should they take?
The concept of microservices has gained significant traction since 2014 when Martin Fowler and James Lewis first defined the term.
In the article "Microservices" published in March 2014, wherein they coined the phrase "smart endpoints and dumb pipes", Fowler and Lewis described microservices as something they were finding "more and more appealing" for developers as an architectural style for application development despite the lack of guidelines on how to properly implement them.
"We've seen many projects use this style in the last few years, and results so far have been positive, so much so that for many of our colleagues this is becoming the default style for building enterprise applications. Sadly, however, there's not much information that outlines what the microservice style is and how to do it." Fowler wrote.
Today, microservices have set new standards in enterprise software development. In a 2020 report from O'Reilly, most organisations were reported to have already been migrating their monolithic applications, systems, and architectures to microservices. Allied Market Research predicts that the Microservices market is expected to grow to $8B by 2026, with a CAGR (Compound Annual Growth Rate) of 18.6% from 2019 to 2026.
In an episode of Coding Over Cocktails, Lewis looked back on the definition he and Fowler had created, and talked about the changes the concept has undergone.
"I think things have evolved in a good way. And the reason I say that is because we're all really looking back at microservices and that definition. People tend to think of microservices in a number of ways," he says.
Lewis says that the definition they came up with for Microservices was like "a meta-paper for a scientific research."
"It was gathering a whole lot of good things that people were doing at the time. You know, it's almost a bit like XP. It's like you take all the practices and you turn it all up to a level in extreme programming. I think microservices is the same."
Lewis then cites Fowler, calling the evolution of the concept an example of semantic diffusion, where the definition of a term starts to stray away from what it had originally meant.
But even so, Lewis explains that there are principles around the design of microservices that remain the same.
"The spirit of microservices architecture is to take good architectural practice and then essentially take it to the extreme with a set of constraints, these principles guiding you."
Approach migration with caution
Organisations have varying reasons as to why they might want to migrate from their monolith systems, such as an expensive license, an unmanageable codebase or that the system is simply ancient.
Whatever the reasons may be, Lewis advises that the migration process be done with caution.
"There are valid reasons and valid business reasons to do these things, but you shouldn't undertake this lightly."
One way Lewis suggests to go about it is by applying certain technical patterns that would work for the underlying monolith system and identifying which parts of the monolith could be moved off.
This technique of doing away with mainframes, according to Lewis, is the "holy grail" among big banking institutions.
"My friends at BBVA who are awesome engineers, they've managed to migrate part of their global payment processing away from them from their mainframe, which is their core banking platform. This is the holy grail of many of the big banks at the moment, and they're on their journey and they've managed it."
Once the migration process has begun, Lewis tells us that it could get overwhelming, as the general overview may look complicated at first glance.
Lewis says that this process of looking at it "too much in the round" is one mistake organisations make.
"Depending on where you're standing, things can look really, really complex. If you just look at them individually, it's a bit like the night sky, right? You look at the night sky, you see all these stars and it looks like there's lots and lots and lots and lots of stars up there," he says.
Although the general overview is necessary, Lewis advises looking at microservices at a smaller scale and grouping business capabilities together and creating teams around them.
"The complexity can be managed by pushing these things into these smaller constructs and having the teams worry about them themselves and worry about the right APIs, and that kind of thing," he explains.
"They don't need to see the whole night sky. They just need to be concerned with their, you know, their solar system."
What’s next?
Being able to witness how microservices and the concepts behind it were able to thrive and evolve, Lewis shares where he thinks evolution in technologies could occur next.
One of the things he put his thoughts on was Serverless. Lewis worries that the unmanaged complexities that he sees with databases might occur to Serverless as well.
"I have a concern about serverless. If you've ever looked at databases from 2001, or big Oracle or big SQL server databases, everyone was going, ‘Ah, all the logic is stored procedures.’ And if you've tried to untangle any of that, where ‘This list is calling that, which is calling this? Which is calling that? Which is calling this?’, it's really, really, really hard," he says.
"I worry slightly that we're going to get into that position with serverless with this."
Despite the concerns, he believes that there’s more evolution to come with Serverless and shares that he wants to see how it would look like in the future.
"We've started to get new things on the scene like Pulumi and CDK, which are much more, ‘I'll write code’ and then something's generated at the back of it. But is that the end? Is that where the industry is going as a whole or is this just a stepping stone to something else?"
Lewis also talks about ethics in machine learning, which he says is an interesting idea they get exposed to at ThoughtWorks and one of the most important topics to emerge.
Particularly, he mentions how having an entirely representative dataset could help with the problem of automated confirmation bias in machine learning.
"If you have a perfect, entirely representative dataset, then yes, that's fine. The problem is we don't have these representative datasets. And this is one side of things. It's not the case that you're getting the right outcome. [It’s] you get the wrong outcome, [but] you can't get the right outcome." he explains.
Lewis hopes ethics in machine learning would improve in order to identify how broad the bias is and the algorithmic consequences it could cause in the future.
"If you've got all the data available, fantastic. If you don't, I think there's real issues."
Listen to our discussion with Lewis on the evolution of Microservices and his thoughts on the future of software development in this episode of Coding Over Cocktails - a podcast by Toro Cloud.
Coding Over Cocktails is a podcast created by Toro Cloud, a company that offers a low-code, API centric platform for application development & integration.
This podcast series tackles issues faced by enterprises as they manage the process of digital transformation, application integration, low-code application development, data management, and business process automation. It’s available for streaming in most major podcast platforms, including Spotify, Apple, Google Podcasts, SoundCloud, and Stitcher.
