Rust, Open Source, Consulting

Drew: How did you get involved with programming?

Matthias: My story is similar to the one that you hear from a lot of programmers. I started getting interested in computers when I was about eleven. I really started by writing my own games. I was lucky enough to get my uncle’s old computer. And, I just wrote code with most of my free time. It was mostly BASIC in the beginning. Usually, I wrote things meant to annoy my brother. For example, I wrote these little quiz apps that I would give to him, but I set it up so that whenever he made a mistake it would make fun of him. Then, I moved on to writing more code in other languages. I wrote some C. I was really excited about anything that would get me working with a computer on a lower level. For example, I learned how to eject the CD, because I could use that to wake my brother up in the middle of the night!

Matthias: In school, I had a couple of teachers that really tried to help me out. They helped me get to the next level. They gave me advanced books on programming and helped me make presentations so I could show off games and things like that. I even went to Ireland for an internship and worked for a company there for a couple of weeks. That was a great experience.

Drew: I think there’s a lot of people who start out just playing around with whatever computer they have at home right?

Matthias: Yeah, and games are an extremely good entrypoint. For example, we used to have an NES. That was our first game console back in the day. The SNES was already out at that point, but we got the previous generation. Anyway, I was always interested in how they built Nintendo games. I never was any good at playing games, because I was just distracted by trying to figure out how they worked.

Drew: So, I guess the biggest question is whether or not your brother ever discovered you messing with him?

Matthias: (Laughing) I usually told him after the prank was done. I actually gave a talk once where I mentioned these pranks. And, I mentioned some things that he still hadn’t figured out. So, he was surprised when he saw the talk. That was a fun time for sure! There weren’t many distractions. We didn’t have the internet, and the book I had on programming was actually the Microsoft BASIC specification that came on the computer. So, it was either figure it out on your own or you had nothing to play with.

Drew: That’s pretty hardcore! How did you even figure out that the documentation was on the computer?

Matthias: I don’t know. I think about that a lot. I think they had an editor, and there was a link to the specification in the menu or something. Or, I may have pressed something by accident and it popped up. I can’t really remember. But, I can tell you that going just from an error to a reference was mind blowing, and I definitely read the entire thing. I just thought it was the best thing ever, because they explained everything about graphics, and they had examples in an example folder.

Matthias: This was all before I even knew simple things like the fact that arrays existed. For example, I programmed this asteroids game where every Asteroid had a name. There were just fifteen asteroids, each named asteroid one through fifteen. So, if one asteroid went out of the screen I would manually reposition it elsewhere. Then, I discovered arrays!

Drew: So, fast forwarding a bit, you now have an established career. Specifically, you’ve done a lot with Rust. So, how did you end up getting interested in Rust?

Matthias: When I went to university, we had a course on parallel programming. One of the tasks was to write a couple of algorithms. They were established algorithms, but we had to write them in C++. Or, at least the majority of students thought they had to do it in C++. But, that wasn't actually in the instructions. Because I knew that C++ takes a long time to write, I thought maybe there would be a way to take a shortcut by writing some of the stuff in Python. So, my love of Python came first. I was the only person who took the exam at the end because I had the necessary points. Everyone else was still struggling with C++, wrestling segmentation faults and so forth. I wondered why the professor didn’t talk about Python instead of C++. That was a glimpse into what happened for me right after, because I joined Trivago on the processing team, which was close to systems engineering. And, I was just able to do so much for Trivago in Python that was really hard to do in C or C++ before. But, at some point you hit a limit with Python where you need to go to a lower level. That was always a tedious process.

Matthias: Sometime in 2015, just before 1.0, I found Rust. I was always a programming language geek, so I learned it just for fun. But, I realized that I could suddenly do more things again in less time. I was able to write something that resembled Python on an abstraction level but that performed like C or C++. It was a revelation! So, I got addicted to Rust and just tried to learn as much as possible. It was like the discovery of fire for me!

Drew: I remember writing almost everything in C++ for my introductory courses. In order to pass each project, you had to put your program through Valgrind for memory checking, and there were always problems. So, I can relate to the pain you described.

Matthias: Yeah, and on top of that, the Rust documentation is really great. Everything you need is in there and described well. They describe things all the way down to the level of working with different operating systems, dealing with different strings, handling error conditions, etc. It helps you learn a lot about the systems that you work with. That really helps you, and it’s such a safeguard to have all of this knowledge encoded in the language. Things will still go wrong! But, they usually go wrong at compile time and you can fix it. In C++, everything is allowed at compile time, but the program won’t really run a lot of the time. To this day, I don’t think I ever wrote a completely correct C++ program.

Drew: Another interesting turn in your career was into open source. So, how did you get involved with open source?

Matthias: When I was at university, I needed a place to store my code. I started with SourceForge, which was a terrible platform by today’s standards. But, back in the day it was the norm. However, there was never a community around code, and I alway felt like that was missing. So, when GitHub came around it was a no brainer to move over. I started to host my blog there, and I hosted some snippets that I wrote for university there.

Matthias: There was a time when I wanted to change my username, because I usually go by the handle mre. That was taken already. So, I sent them an email, and I got a reply! They gave me the username! I was so impressed by that turnaround, and it felt like it really was a startup. The email came from one of the founders who ended up becoming the CEO at one point. That goes to show how small the team was back then.

Matthias: Nothing else really happened for a long time until I started to host some other projects on GitHub. At one point, I had a small rock band, and I wrote a CMS system that we used for the band. I also wrote a static site generator for invoices because I hated writing invoices. So, the generator would take a markdown file and generate a PDF. Whenever I had something interesting that I worked on, I usually defaulted to pushing it to GitHub and seeing what happened. Lots of things that I pushed didn’t get any traction, but some of them did. It was nice to meet people there and shape a community around products.

Matthias: I have so many cool stories about open source development. For example, I had a script that would fetch images from Dropbox and put them in a website. So, it was a dynamic gallery of sorts. I created that for my band and forgot about it. Eventually, the API was deprecated and the project needed work. Hacktoberfest was coming around, so I created a couple of issues just to see if anyone would take on updating the project to work with the new API. It turned out people did! One Christmas, I got an email from someone who said they put my dynamic gallery on a tablet and gave it as a present to the grandparents so they could see photos of the kids. This was a use case that I never anticipated, but it makes me extremely happy to know that this runs on someone else’s tablet now and makes someone happy everyday!

Matthias: Another story is about a platform for static code analysis tools that we started. In the beginning, it was just a list of tools that I found in a markdown file. Eventually, the list of tools grew and we needed a renderer. Soon enough, we had a website for it. It’s called analysis-tools.dev. When the pandemic hit, I got a message from someone from Kenya who was having trouble paying rent. She asked me if she could work on this project, because she knew that there was an open collective and we had some cash that we wanted to use to compensate contributors. So, I said sure, just send us an invoice when you’re done! She came back with nice work, and it helped her out during that tough time. Eventually, she applied for a position here in Germany and works as a software engineer now. To me, that story just shows the power of open source.

Drew: That really is great. I actually meant to ask you what you had seen in terms of being able to make money and support yourself in open source. Do you feel like that is a viable career path right now?

Matthias: Going full time open source would be hard. And, if you do that, you can’t think like an open source maintainer. You have to think like a business person. Even though you write open source code, you have to focus on building something that people will want at some point. You might think of open source as a nice platform that you can build abstractions on top of. With my experience, the projects that took off were usually the ones that people could immediately use for themselves and then later find a use for at work. It would have been extremely hard to know early on what would have been a successful commercial project. Overall, It is probably easier to join a company and do open source on the side or as part of your job. It’s a lot harder to start just in open source and then later build a community and find businesses to support your projects.

Matthias: One thing that you can always do as a developer is start very small by building something that you want for yourself, and then listen for feedback. You might ask people things like what they’re using the code for, what was the pain point that made them star the project or contribute to the project. A lot of people that star or contribute have a problem they’re trying to solve for their business, and this is really great context. If you’re open to talking to these people, they’ll usually be open to talking to you. You might start a consulting relationship where you do something like add features for them. Eventually, they might have you join the company or become a sponsor for your project. That’s all very abstract, and you’ll have to get into the details, but from my experience it’s very hard to pull off. I will say that the projects that have worked for me are those that are user-facing somehow, and not the ones that are more low-level or infrastructure.

Matthias: Another thing that people might try is taking an open source project and creating a hosted version of it. In that case, I would focus on the marketing side first and see how easy it is to get sales for the project. This teaches so much about how businesses work that you start to get a better feel of which projects might work. So, just think of it as an entrepreneur and start from the customer perspective.

Drew: That’s all really interesting to me. I don’t know why I hadn’t thought about that before, but there really is so much similarity between open source and entrepreneurship in terms of figuring out what to build. Do you feel like you have a good sense for when a product should be open sourced versus just being a closed source business? Is there a clear line there?

Matthias: If you build things that are very specific to your company, then maybe it makes sense to keep it closed. One of the things I like to do is think about “inner source.” So, you might be developing a closed source solution, but the documentation, the testing, and the organization of the codebase is as-if it were open source. This can help you later to just flip the switch and make it open source.

Matthias: Another thing I would say is, if your code is not your product, why keep it closed? I think this is especially true if you build tools for developers. For example, I built a tool called Lychee, which is a link checker written in Rust. I didn’t build it to be successful. I just wanted something that would check links for my projects, and it turned out that other people needed something like this too. But, the code itself is really not that interesting. Even if you build a platform on top of this, it’ll still be boring. But, no one cares about this code. They just care about checking links. So, they’re going to contribute what they need to an open source project rather than build something new if possible. This is why I think this can be open source. The companies that wouldn’t want to pay for a hosted solution wouldn’t be the right customers anyway. So, that’s how I’ve been thinking about it with that project.

Drew: That’s insightful. One last question on open source. If you hadn’t done any open source at all, what would be your first steps getting into open source?

Matthias: A lot of people think that it’s too late to start with open source because of how many developers and projects are already out there, but I can tell you “it’s still day one.” The majority of developers still don’t do open source. There are more and more people who join each day. Doing open source is a good habit to have just like picking up running or any other sport. You don’t do it because you think you’re going to win a marathon or be the best immediately, you do it because you believe in doing it. And, if you make it a habit, maybe eventually it will become more than just a habit. It might become a passion or something you could put on your CV. Looking at it and feeling overwhelmed is normal, but you just have to remember that every project started with a single commit. And, start small like I mentioned. Try to find the smallest problem, refine it to something even smaller, and then try to fix it and fix it well. When you do this, you find that nothing is extremely well done. There are a million libraries out there that are missing major functionality or never even got finished. And again, just start really small. Don’t think “elastic search,” think “a cli tool that I need today.” Also, you don’t have to go all in on open source. It is something you do for the long run, and chipping away at it little by little, the interest will compound. It can get to a point where you’ll find that you’re better than anyone else at one little thing just because you’ve been chipping away at it.

Drew: That’s great advice. I did want to ask about the consultancy you’re running right now called Corrode. How did you decide to start Corrode, and how did you start it?

Matthias: Around 2020, I realized that the Rust community was growing, and not only linearly but almost exponentially. There were tons of new companies entering the scene and lots of exciting things going on, and I decided that I wanted to participate in that. I wanted to help grow the ecosystem, because I believed that something like this really should exist. I knew that there were already other consultancies, but I thought there was still room for more. And, each little consultancy can focus on their own niche. My niche is in idiomatic Rust and web applications. This is something that I like to do and know how to do well. And, I really care about code quality. That ties into things I’ve done previously like analysis-tools.dev.

Matthias: Once I realized that I wanted to work on accelerating Rust adoption, it was clear that this wasn’t something I could do by working for one company. I needed to be some sort of advisor for multiple companies. So, that’s what I did. And, as It turns out, companies aren’t always thinking about Rust when they come to me. They usually have business goals in mind and legacy code that we need to work around. Rust isn’t always the perfect solution for them. It’s more about figuring out the problem they’re trying to solve and the capacity of the team to solve this problem in Rust or some other way.

Matthias: Consulting is not about selling people a solution that you have in mind. It’s more about listening to the customer and thinking about how you would solve the problem if it were your business. Rust adoption is something that a lot of companies do very carefully. Usually, you have two or three people within a company that are really excited about Rust and they want to push this project forward, and I have to understand how that fits in with the bigger goals of the company. I sometimes end up playing mediator between the development team that is excited about trying something new and the reality of day to day business.

Drew: Do you find that the companies you work with tend to have a similar set of challenges when they come to you?

Matthias: There are multiple types of problems, but I can cluster them. On one end, you have big, established companies that have highly optimized production workloads and run Rust in their infrastructure. They might also have other languages that they’re struggling to scale. Usually they’re attracted to Rust for performance reasons. But, usually when we dig into it, performance isn’t really the problem because most of these things are already optimized or can be scaled horizontally. The true problem often ends up being churn and toil, all of the things that make site reliability engineering so tedious. They have on-call rotations, and things fail. They have segmentation faults in production, they have unmaintainable code. Maybe they’re struggling to add concurrency. So, the big benefit for these companies ends up being stability, not performance. For these companies that need something really reliable, Rust’s stability is most important. It’s nice to help these people get some rest in the evenings and not spend everyday firefighting.

Matthias: The other end of the spectrum is people in the startup world working on greenfield projects. Usually, they have a couple of people who are already working with Rust or have tried a few things and now they want help getting buy-in from management. I usually find that they’ve already done a couple of really great things, and usually management really likes the passion that these people bring. Management usually just wants to understand the risks. And, there are a couple of supply-chain risks with Rust. For example, there are some missing enterprise features in Rust, there’s not a great, private crates repository with adequate security. These are the things that I highlight. So, there’s two jobs with these companies. I talk to management to help them make the decision, and I work with the developers to improve the code they write. Often, this means introducing some software design patterns and showing them how to implement those patterns in Rust. So, for this group of companies the biggest thing is introducing more architecture and good practices to get their development teams to the next level.

Drew: Where do you see Corrode going from here?

Matthias: It would be nice to collaborate with other people. I’d like to have other people who are equally passionate about Rust but that also can bring the bigger picture thinking. They need to be good at understanding where Rust fits in and not just jump to rewriting everything in Rust. And, I think this could be modeled after open source development. Usually, for every thousand or so people who visit your project, there are one or two who are passionate and skilled enough to contribute. And, sometimes they do. So, it’s important to make it as inviting as possible for people to contribute. For Corrode, that means that if I build training material, it’ll be open source. Or, if I am onboarding a new client, I might reach out to someone that has expertise that complements mine to help. And, like I said earlier, there’s room for multiple consultancies. Every consultancy is different. There may also be room to say no from time to time and point a client to a better consultancy to fix the problem. There’s really not a lot of competition right now, because there’s still way too few consultancies out there.

Drew: I like the way you talk about growing the ecosystem and creating more opportunity for yourself and others. I think there’s always a temptation in business to think about things as zero sum. But, your approach seems much better.

Drew: Do you have anything else that you might want to share with our audience?

Matthias: I think as a community we need to be very aware of our outside perception. We seem to be at an inflection point in the history of Rust where it could still go either way. Historically, Rust has been the most loved programming language. But, it feels like we’re moving toward a different level of maturity. What companies will expect is a stable foundation, and a reliable platform they can build on top of. And, achieving that includes managing the community’s perception. For me, that means making Rust’s great features shine, and not thinking too much about politics or getting into “religious” fights over which programming language is best. It’s really about excelling on the engineering front and minimizing the drama. The drama has become a distraction from what Rust really can be. We have to be aware of that and work towards creating an ecosystem that is really inviting.

Drew: That’s a great thought to close on. Thanks for all the insight you shared.

Matthias: You’re welcome!