Polyglot programming, whilst not a new concept, is becoming increasingly relevant to full stack developers and to microservice architectures. That’s because it allows a developer to mix and match programming languages to suit their personal preference, expertise or the functional requirements of the project.
ThoughtWorks Director and Software Architect Neal Ford joins us in this episode as he discusses the benefits of polyglot programming, a phrase he originated himself. He also shares how developers can use this practice in making application development more efficient, and how the concept of low-code is relevant in all of this.
Kevin Montalbo: In a developer profile survey conducted by Stack Overflow this year, 55% of the respondents identified themselves as full-stack developers. This represents a 7% increase over the last two years alone. It’s apparent that developers are becoming proficient in more programming languages or are using technology stacks that cross the boundary of both frontend and backend development.
Today on Cocktails we discuss polyglot programming with the person who originated the phrase itself. We learn how polyglot enables a developer to express his or her creativity, how it is a natural fit for microservices, and where low-code fits into all of this.
Hello internet, my name is Kevin Montalbo. Welcome to another episode of Coding Over Cocktails.
Joining us today, all the way from Australia, is of course David Brown. Hi, David. Good to have you on.
David Brown: Good day, Kevin.
KM: Our guest for today is a Director, Software Architect, and Meme Wrangler at ThoughtWorks, a global IT consultancy with a focus on end-to-end software development and delivery. He is also the designer and developer of applications, articles, video presentations, and author and editor of an increasingly large number of books spanning a variety of subjects and technologies, including the most recent Presentation Patterns. His professional focus includes designing and building large-scale enterprise applications. He is also an internationally acclaimed speaker, speaking at over 300 developer conferences worldwide, delivering more than 2000 presentations.
Joining us today for a round of cocktails is Neal Ford. Neal, it’s a real privilege to have you on the podcast.
Neal Ford: Thank you very much for having me, I’m happy to be here.
KM: Let’s begin. In 2006, Martin Fowler, your friend, wrote that you coined the term polyglot programming to express the idea that applications should be written in a mix of languages to take advantage of the fact that different languages are suitable for tackling different problems. So can you tell us the process that led you to coming up with the term polyglot programming?
NF: Well I think it's one of those things that's kind of obvious in hindsight, because if you go back to 2006, when I came up with that, that was really the beginning of the Cambrian explosion of languages supported on both the JVM and .NET.
You know, .NET was specifically designed to support multiple languages, but the JVM came out strong, too. It was a bit of engineering brilliance, I think, on the part of the JVM to separate the runtime from the language and have basically an assembly language that runs the runtime because it allowed them to innovate on the runtime without affecting language and vice versa. Suddenly, you had languages taking advantage of that runtime that works natively in Java. And I started to, I've been a language geek forever. That was one of my specialties when I got my computer science degree was languages and compilers.
I've always loved computer languages of all kinds. And it fascinates me how they're alike and how they're different. I noticed that this was part of the beginning of the rise of tools like Ruby on Rails, which is kind of the mid-2000s version of the rapid application development tools from the mid ‘90s to my first book was actually a Delphi book, which is a rapid application development environment, but way back in the nineties. And there were a lot of similar aspects to Delphi and 4GLs and the low code environment stuff that we see now. The rise of tools like Ruby on Rails, but you've seen even more pervasive versions of this with tools like Gradle, for example, which is a domain specific language written on top of Groovy that has become sort of the universal bill, certainly for Kotlin and a lot of the Java ecosystem as a replacement for Maven.
And it was an observation that a lot of these new languages, particularly dynamic languages, give you a level of expressiveness that's just not possible in static languages like Java and particularly around things like user interfaces. So, I don't know if you ever wrote any swing user interfaces, but there was just an enormous amount of declarations for types and all, and it's like their user interface. I don't care about the types and the compiler yelling. I just want to build quick and simple user interfaces. And a lot of these other languages allowed you to do that more freely. And because they all compiled down to byte code, it made me realize that, “Hey, there's a common language here that you can, maybe for the first time, really effectively mix and match languages.
And so let's talk for a second about computer languages. I've always been fascinated about computer languages and because computer languages, it's an engineering medium, but it's also a creative medium. There's a great debate in the programming world about static typing versus dynamic typing, which is almost as fraught as the VI vs Emacs or Eclipse vs IntelliJ argument. And in that, there's never going to be a resolution. What made me realize that? I've met Martin Odersky, the person who created Scala. I've also met and chatted with Rich Hickey, the person who created Clojure. Scala is a strictly, strongly typed language with static typing. Clojure is strongly typed, but dynamically typed, if you take Martin Odersky and make him write Clojure, he's miserable. If you take Rich Hickey and make him write Scala, he's miserable! But if you allow them to work in their own medium, they're incredibly productive. And what that made me realize is computer languages can't be objectively evaluated, because it's a creative medium.
Every person finds the creative medium that they like to push against that allows them to be creative. There's a great quote that I bring up here from William Faulkner, Nobel prize winning novelist. And he very famously has a quote that said “I started out as a poet and I was terrible as a poet. So then I tried writing short stories and I was terrible at that too. So I guess I'm a novelist.” But he discovered what his kind of, the real secret to creativity is not freedom. It's constraint, it's the right kind of constraint to push against. So, I have a strong preference in computer languages for dynamically, strongly-typed languages like Clojure or Ruby. I find that I hate arguing with compilers, particularly when I know I'm right. And the compiler is just being a bureaucrat yelling at me about something it's like, “I got to typecast this again, [but] this is the right thing! Stop yelling at me about that!” That's frustrating to me. I'd much rather write tests and validate that way than have a compiler, but there are a lot of people who love having the compiler to argue with and, you know, to validate what they're doing.
And so, it made me realize that there is no one true programming language. Developers should be able to choose their own. And if you extrapolate from that, there's this idea that, well, do we have environments now that would support this idea? Now you can't freely mix and match because you know what, the fundamental level, there's still some differences, but it does open up the door for, you know, a team to write Ruby on Rails on top of Java, you know. J Ruby was a big thing in 2006, where you can actually deploy your Ruby code on the JVM, and that allowed you to be a Ruby developer, but still take advantage of the good engineering behind the JVM. And so there's a very long explanation for, you know, I realize that, you know, it is maybe for the first time ever possible to allow your creative muse to run wild as you run programming, but still have good engineering solutions that have a fighting chance of being able to integrate with the stuff that other people on your team are also writing.
DB: You talk about a creative preference in terms of a language. What about like a functional preference where a language is just better suited to a purpose, you know, polyglot?
NF: There’s certainly cases of that. So, the only person that I know that has Haskell code in production, and that's a pretty rare thing, cause Haskell's a great way to think about things, but man, you do not want to put that in production and argue with, like IO and stuff like that forever. But our CTO, Rebecca Parsons actually talks about some code that was production-ready that she created in Haskell. And she said, you know, they were doing like a single signal transformation process and all this really heavy math. And it was like a hand-in-glove fit for that language. And so there's a lot of the affordances the language offers you to get stuff done. At some point they're all “Turing Complete”. And so, you know, anything you can write with assembly language, you can write with Erlang or Ruby or C sharp, but it's how difficult it is to write that and what affordances does it give you. And I think that's the thing that developers and the creative side of developers, you know, software development is this weird mix of creativity and engineering. I think that's the real appeal to software development because it's almost equal parts, you know, artisan versus engineering and finding the way to express the artisan part of yourself while supporting the good engineering stuff, I think is a quest that every developer goes through. And if you're lucky enough to have utilize a whole bunch of different programming languages, I think you finally settled in on the one that you feel most comfortable in. Certainly, for me, my two favorite languages now are Ruby and Clojure for very different reasons. I use them for very different things. So, Ruby for me is the language that provides the most developer happiness almost to a pathological degree.
So, Ruby is so helpful that it will help you write into a snake pit full of vipers and all sorts of bad stuff. And it will gladly help you ride along the way because it's trying to be as friendly as possible and offer you all these affordances. Clojure is trying to maximize engineering happiness. So, it actually tries to get you to avoid bad things like mutability. That's discouraged strongly in Clojure because they've realized that, “Hey, that's kind of bad”. So, for building really big, well engineered things, I think Clojure is awesome because it gives me strong but dynamic typing. It's also a Lisp which has some distinct advantages, but it's not nearly as friendly as something like Ruby. So, when I write little scripts, I still use Rake all the time, the Ruby build tool, because it uses the Ruby language features but in this beautiful, almost transparent way.
So, I actually wrote some Rake code today for the book project I'm working on because the thing that checks in my book stuff is a Rake file. But if I really want to build something that's very well engineered, I'll choose something like Clojure. So, I think it's really contingent on every developer. And in fact, it makes me sad when I see developers who've really only used one language. In fact, I'll tell you the killer interview question if you ever have to interview someone from a programming job, I do this for every interview I do for a developer. One of the questions I ask them is: “So, let's say you claim that you're an expert in Java. Okay. So now you've been made king of Java, which means you can add any feature you want to the language, but in doing so you also have to remove one of the existing features that you don't like. Go!
I want to see the thought process, because if you've only ever worked in Java, you'll just freeze up like a deer in headlights, because I can’t imagine adding or removing anything because it's the only thing I've ever used. But, if you've been in a dozen different languages, they immediately kick into, “Oh, wow, what would I, ih, generics you gotta fix that. And, you know, serialization, you got to get rid of that”. But then, and so you see the thought process and it really shows you how polyglot that person is because every programming language you learn, you learn some new aspect of programming and you take that into the programming language you're currently using all the time. Everybody here’s had the experience of, if you transition from Java to Ruby or Clojure Scala, the first code you write is just really bad Scala code that replicates the Java code.
But as you learn the idioms of Scala, you become more of a Scala programmer. But then when you go back to Java, you take a bunch of those idioms with you and you're a more effective Java developer. So, I believe that the more polyglot you are, the more effective you are in your native language, whatever it is, because you've seen lots of different ways to solve problems. And it gives you more tools in your toolbox to reach for, in terms of abstractions and patterns and capabilities that you might not have had before. The big shift of course, is the functional programming shift that all these languages have gone through. Java was the last big one, but now we have streams. We have “map” and “reduce” and “filtering” and all those things, which is just another really nice set of tools in a programmer's toolbox that while I can now can use a ration. I can use a design pattern or I can use map reduce all within the same language. And if you have got experience in several languages, you know how to better use those tools in your toolbox.
DB: Let's talk about architecture. The polyglot programming seems to be a natural fit for microservices, where you can have multiple teams working on a project, independent project, services-orientated architecture, where you're designing services in independent teams, they're using their languages of choice, whether it's a personal preference or because they find that programming languages fit for purpose. Is that what you're finding, is microservices a natural fit for polyglot programming?
NF: It is because there's such a level of isolation. It's based around this idea of bounded context. There's a great story from the early days of microservices from a well-known to-do list application called Vendor List. And Chad Fowler was the architect on that. And it was one of the early microservice success stories. And one of the things he did was mandate that every service be written in a different language, which is a bit extreme. But, one of the things that you're trying to achieve in microservices is no incidental coupling between their services, no shared stuff, because that breaks the bounded context and one sure-fire way to make sure nobody's actually sharing anything is make somebody write in .NET and somebody else write it in Java. There's no way you're accidentally sharing class files across those. And so it forces that level of decoupling. But the real benefit of that is that it allows developers to not over-engineer things and use a language, a database, a platform that's more suitable for the problem. One of the anti-patterns we commonly see in large organizations is this idea that, “Hey, we're going to standardize on a single technology stack because that'll be cost savings”. So, when you think about governance and architecture, two philosophies: software is either overhead or it's strategic. What do you do with things in an organization that are overhead: you minimize costs. Because the more you minimize the cost of overhead, the more profits you make.
And so one of the failed strategies that people do is, well, one of the ways to manage overhead is to standardize on single platforms and single technology stacks. So, we'll pick Java and Oracle and MuleSoft, etc. But the downside of that is, okay, so now let's say you're a responsible enterprise architect and an organization like that. And you only get to pick one relational database. Well, you need to go find the team that has the worst, awfulest, nastiest relational database problem. And you've got to pick a tool that's powerful enough to solve that problem. The problem now is you now force that complex tool on every team because you only get one relational database. If you're like most organizations, most of your projects need a dumb place to store stuff and get it back. But they're dealing with the complexity of an industrial strength database. If you only have one option in your enterprise, you always have to pick the most complex thing and now everybody is using the most complex thing. And the irony of that is the motivation behind that philosophy is cost savings. And now you're over-engineering everything to the point where you're just burning through money because you're using these crazy complex technology stacks for very simple problems. One of the great revelations in microservices is like STOP! Simple things can have a simple tech stack, PHP, MySQL database, done! We'll get it done in a week and we're over! Versus Java, Maven, Oracle, enterprise service bus – you know, the whole technology stack.
The great revelation for this to me was a big, enterprise client we're working with at some point. And we were asked, I'm a consultant by trade, we're asked to come in to solve this problem. And it was the quintessential mid-early-2000s architecture where it was a J2EE, they were using Cocoon, which was this publishing framework and this giant enterprise service bus. And I was brought in because they were having problems because their application server was getting stack traces because of thread problems and a bunch of other stuff. And at some point somebody said, “Yeah, you know, and all this stuff is here. And you know, the 12 users don't like this very much”. And I said, “12 users, what are you doing?!” And it was literally this thing where you filled out this wizard like four pages and then clicked, “Okay” to get a project code. And they had built this thing like it would launch a space shuttle. And I said, “Look, I'll go home this weekend. In Ruby on rails I’ll write the whole bloody thing in like six hours and we'll be done.”
And we did. And that was the revelation for them. It's like, “Oh, we have over-engineered everything to a ridiculous degree”. And if you only have one thing, it drives this massive over-engineering. And so, part of the giant benefit of microservices related to this idea of polyglot programming, is it freed organizations to say “let's right-size the technology choice for the size problem we have”. Now, this is not advocacy to go crazy with this and we got a thousand flowers bloom. And so, in our Building Evolutionary Architectures book, we actually introduced something – and this is [one of the] hazards of writing a book. We introduced something that we call “Goldilocks Governance”. And then we realized after we published the book that only people in Europe in the US know who Goldilocks was. Nobody in Asia or anywhere else knows the Goldilocks fairy tale. But of course the idea was finding the “just right fit” for these things. Not too much, not too little. And the idea of Goldilocks governance is let's pick a few technology stacks – a simple one, a medium one, and a complex one – and standardize on those rather than one size fits all or, you know, one unique one for everybody. And I think microservices really facilitates that because it does have that extreme level of decoupling and the idea of bounded context, which keeps everything local from a technology standpoint. And you can integrate with things as you need to, without going down that over-engineering route.
DB: How does the paradigm of low-code application development fit into all of these and the paradigm of design patterns?
NF: So, design patterns are interesting, in that, they've slowly been eaten by programming languages. So, if you look at the original design patterns book, I actually just did talk at a conference in India last night. It was morning for them but night for me because that's the way things work now. It was modernized design patterns, because if you look at the original Gang of Four book, which was written in the early ‘90s, the two exemplary languages in that book were C++ and Smalltalk. Now there's still a fair amount of C++, and it's undergone this revolution – it's a really modern programming language now, but not the version from the mid ‘90s. And a command design pattern is a great example of that that is built into every language you'll ever use now, in terms of Clojures. As soon as Java finally built Clojures into the language, the command design pattern literally melts away because it's now a feature of every language you ever use.
Many of those design patterns have been slowly eaten by facilities that are native to compilers. And in fact, there's a whole new set of what we now call design patterns that come from the functional programming world. If you look at capabilities like MapReduce, well, if you expand that into architecture, you have Hadoop, which is a big giant MapReduce framework. And so we're seeing new patterns show up that are sort of the next generation of design patterns, because in many ways the design patterns book should have been called making C++ suck less.
But now we don't have a lot of the constraints in the languages because the languages are growing toward the expressiveness we need. The latest, sort of a board-like, simulation of all these functional programming concepts and all the mainstream languages is a great example of that. We’re borrowing all these patterns from the functional programming world and adding them to pretty much every language that's out there.
Now, the low-code thing is very interesting. I'm a member of the ThoughtWorks technology board and we produce the ThoughtWorks Technology Radar twice a year. And we just came out with the most recent edition within the last month. And one of our themes was a theme called democratizing programming, which is exactly about these low-code environments coming out now. Now I'm an architecture architect by trade. And I view there's nothing in my world that is pure advocacy because everything is a trade off. My most recent book was Fundamentals of Software Architecture, and we codified our first law of software architecture as “Everything in software architecture is a trade off”. And the first corollary to that law was, if you think you found something that isn't a trade off, you just haven't identified what the trade-offs are yet.
And that's the hazard with low-code environments: people go in and they only look at the capabilities, but they don't look at the trade-offs. And so, they're useful for a lot of things, but then people believe that they're useful for everything. So, one of the things that we codify in my book Building Evolutionary Architectures is this thing that we call the last 10% trap, and every low-code environment suffers from this. So, I'll describe this and then apply it to the modern world. So, this revelation came to me because I used to work for a consulting company that did application development for clients in Microsoft Access. And we ended up shutting down that division of the company because we realized a common pattern kept reoccurring. That every access project started as a booming success and ended in total failure. And we had to understand the dynamic of why that kept happening over and over again. And we codified this as the last 10% trap. So, using something like Access, the first 80% of what the user wants is amazingly fast, and it's spectacular how fast you can get there. The next 10% is possible, but difficult, because now you're trying to do something the tool wasn't really designed to do, but you can kind of bend it and hack it and figure out work arounds and other stuff. But that last 10% is impossible and users always want a hundred percent of what they want.
DB: Well, this is a really interesting point, and this is what, we're often telling people is the difference between a low-code solution and a no-code solution is, is that critical last few percent where to get the job done, you really need to code to be able to get there and build it is whatever that custom function is or custom workflow or whatever it is that you're trying to build is you ultimately want to be able to write some sort of script to some sort of a library object, whatever it might be, that you can call into your, low code application. We're talking about low-code, we're not talking about no code.
NF: It's a matter of degrees because you remember, do you remember the 4 GLs? Access, dBase for windows, Clipper, Power Builder. Where are they now? Those were all low-code environments. They all suffer from the last 10% trap. So, I will use your low-code environment if you will give me the following things. I want the ability to diff it. I want to be able to version it. I want a first class IDE that gives me refactoring support. I want to be able to test it at the unit, functional, and user acceptance level because I get all those things with code. And so if you're going to make me go to a different low code environment, I want all those things because that's a trade-off, and I might not want to trade those things off because you know exactly where I am going to go.
Here's the thing that I've realized most recently, if you are going to choose a low code environment. So, your normal architectural approach when you're using something like that is to build the simplest, front to back vertical slice to make sure it works. That is NOT the correct approach in a low-code environment. The first thing you want to do is do the most difficult thing because you know, the easy stuff is going to be easy. If it's not, then what are you doing? You need to make sure how far you can push it, to see where that 10% is going to fall. And so the first thing you should do is the most difficult thing, because that's where you're going to find out that the transactionality doesn't work or the scalability doesn’t work, or you can't get enough access to the underlying data to make it work.
That's the way to vet a low-code environment is not the simple “Hello World!” because if that's not simple, then you've picked the wrong low-code environment. It's the really complex things. Because the first thing you need to find out from an architectural standpoint is how far can I push this? And now that becomes your decision criteria. This is not to say you shouldn't use it, but you should go to your stakeholder and say, okay, “We've experimented. We can do 93% of what you want. Are you okay never having that last 7%, because if you're not, then we can't use this environment. We got to choose something else. If you are, then we're going to go into this wide open. So when we get to 93% and we're done, you can't say no, no I want the next seven. It's like, no. We told you way back when this is what you're going to get”.
So, spreadsheets are the perfect example of this because if I look at a spreadsheet as a programmer, what I see is a real time IDE with functional programming built in. It's a matrix manipulation just turns out accountants are matrix junkies. They just don't realize it. And the way that spreadsheets have been so effective is people understand their capabilities, but they also understand their limitations. So you don't try to push a spreadsheet further than what it'll go. You realize that, Oh, I can go this far with a spreadsheet, but then I'm going to hit the wall. But that's okay because I knew that going in, you've got to apply the same rationale to these low-code no-code environments that are all the rage right now, because it's exactly the same decision criteria. Can I live with the compromises this is forcing on me?
DB: I think we're totally on the same page. You know, our approach with Martini was to empower developers with low-code, not restrict them. We're targeting the enterprise class market. With enterprise problem sets, you need the power of code when you need it. We'd looked at empowering them in terms of making them more productive, getting rid of these cookie cutter functions that they need to do over and over again particularly for application integration or APIs, or creating services and having them automatically restful or whatever it might be. Consuming data, transforming data, these are a lot of cookie cutter functions, which can be cut down into a low-code environment. But when it comes to that proprietary function for your particular business process, it may be that you need to write some code and email to slot that code into that low code environment.
And that's exactly the process philosophy we have taken with our low-code tools is that we're not trying to put a firewall on the developer in terms of the scope of the application that they can build. The objective is to make them more productive. As a result, the solution is not designed for a citizen developer because ultimately you're getting that blame where having a low-code environment, you either go for the citizen developer and make it no code. And you're going to have those limitations. that Excel spreadsheet type limitation, which can be perfectly fine because I have a very wide market as spreadsheets do. But, if you're targeting a professional developer, then there's no point putting those boundaries on them.
NF: There's a spectrum here. So, almost everything in software is a spectrum too, which is why I believe we keep going back to fourth generation languages and these fundamental architectural styles like microkernel, modular monolith, microservices, because those are indeed flexible enough to solve any kind of problem in any level of abstraction. We try to build above that always has some sort of limit built in, and I think it's the job of an architect. So one of the things that I try to instill in junior architects at ThoughtWorks is, don't become a blind advocate for anything, because by definition, if it's an architectural component, it has trade offs. And so your real job as an architect is “Can I objectively evaluate the trade-offs on either side of this decision and make an informed decision going forward and realize that some of those decisions are in fact non-reversible”, but you're okay with that because using a low code environment will get you to the intermediate finish line or close enough finish line faster. And that's more value to the organization to build it in a more traditional kind of software stack that will take much longer, but you have that ultimate degree of flexibility because IT moves so fast now.
It's often very strategic to choose an environment that gets you to 90% there. And don't worry about the last 10%, just like a spreadsheet rather than spend the time to get all the way to, you know, a hundred or 99% using something more traditional. So it's always about evaluating trade-offs objectively. One of the things that I tend to say is I try to never let architects use words like “best” or “worst”. It's a different set of trade-offs. That's the thing you have to evaluate as an architect is what are the real trade-offs here and then make an informed decision going forward. Because in many cases, I try not to make architecture decisions. I try to evaluate trade-offs and go to stakeholders and say, okay, here are the menu of trade-offs pro and con you decide, because it's your money, you decide. And then we will build as much as we can in the approach that you pick, knowing that we have these pros and cons going in.
DB: Look, I think we could keep on going and going. There's so much to talk about, and there's a whole bunch of topics that I would love to have addressed with you, which I just don't think we're gonna have time for, maybe in another series we can talk. I noticed, for example, you have a training program on a services-oriented architecture where the microservices, and now the ultimate iteration of SOA and how it fits into the services based architectural styles and how ESBs fit into all this and all sort of stuff. So I would've loved an opportunity to talk to you about all of that.
NF: We can always do it again!
DB: Neal, how can the listeners find out more about you, the books you've written and follow you?
DB: Amazing. Congratulations, we look forward to talking to you about that in the future. Neal, thanks very much.
NF: Great. Thank you. Enjoyed it.
KM: All right. That's a wrap for this episode. Thank you very much, Neal Ford, for being with us. How about you, listener? What are your thoughts on polyglot programming? Do you have any stories that you'd like to share? Or any podcasts, guests and topics you'd like us to cover? Let us know in the comments from whatever podcast platform you're listening to. Also please visit our website at www.torocloud.com for our blogs and our products. We're also on social media, Facebook, LinkedIn, YouTube, Twitter, and Instagram. Talk to us there because we listen, just look for TORO Cloud. Again, thank you very much for listening to us today. This has been Neal Ford, David Brown and Kevin Montalvo at your service for Coding Over Cocktails.