Building Search In Rust

Drew: How did you end up working on Meilisearch?

Kero: I don’t know when I started programming, but I was a teenager. I was programming in the PHP framework Laravel at the time. My first big project was a sort of social network. Yet another one (haha). It was very bad. It relied on a lot of APIs like the YouTube API, Twitter API, and so forth. I also don’t have the code anymore because I wasn’t even using Git. I was just basically copying and pasting stuff around on servers. The website was crude but impressive in its functionality.

Kero: After having that taste of programming, I enrolled at the 42 School in Paris, which is a programming school founded by Xavier Niel, the founder of Free here in France. I was one of the first students there. I think I was there for something like five years. Typically, people graduate from 42 in three years, but I spent the last couple of years redoing the previous projects in Rust. I just really wanted to improve them and have the best programs I could. I even beat the staff at some of the internal competitions by doing things in Rust! I’d actually like to write a blog post about that experience one day…

Kero: After graduating, I started working at a big ecommerce company called Veepee. I was in a research group with the goal of solving internal problems for the company. At one point, they wanted to improve the search engine. So, that’s where Meilisearch begins.

Drew: You’re one of several people I’ve interviewed at this point that is working in Paris. Paris must be a much bigger tech scene than it gets credit for.

Kero: Well, a lot of AI companies are moving to Paris right now. Mistral is probably the most famous of those. But, there are a lot of companies moving to Paris. I think it will grow a lot in the future months.

Drew: It seems to me like Paris might be the most prominent European city for tech.

Kero: London is very strong, and I think it has been London. But, I think you could say that more and more Paris is taking the lead. For example, we did our first funding back in 2019 in London. The big investors wanted us in London, and raising in France was hard. Now, I think you’d have an easier time raising money in Paris.

Drew: So, I was interested in what you started talking about in regard to Veepee wanting to improve their search engine. One thing that has always shocked me is how poor the in-site search experience is on a lot of websites. I pretty much stopped using those built-in search bars because they’re usually so bad. Is that what the problem was at Veepee?

Kero: Yeah, exactly. A funny example was that searching for red wine on Veepee would often return red dresses even though there was plenty of red wine to be found on the site. The relevancy was very bad. And, that was six or so years ago. Unfortunately, even today there are problems. Just the other day I searched for white shirt and it showed me knives with white handles.

Drew: It’s so frustrating! Why has this segment of search been so bad for so long?

Kero: Well, a lot of companies use Apache Solr, which is basically the non-Elasticsearch option. I don’t think it is very good. On top of that, companies often don’t know how to manage their search engines well. It’s also probably an issue of how they’re categorizing documents. It’s hard to say exactly why. The answer is probably different for different companies.

Drew: So, you were at Veepee and they had this problem with search. What happens from there?

Kero: I was an intern in that internal research group and eventually got hired. But, I still spent a lot of time in that original lab talking with the people there. So, they were working on the search engine without me, but I was very interested in it and followed the progress closely. One of the choices they made was to use Golang. That’s not a bad idea, especially because the group of people maintaining the engine had a lot of turnover, and I think Go can be good for picking things up fast. But, I felt strongly that Go was not the right direction for the performance of the engine. Obviously, my main concern was the garbage collection. There’s a lot you end up having to do to try and avoid running into garbage collection. So, I suggested they use Rust. They didn’t take my advice, so I thought it would be fun to work on my own search engine over the weekends. At first, my goal was to basically achieve parity with Algolia, delivering similar performance and behavior. What I ended up with actually seemed a lot better than the one done by the lab team. Initially, I called the engine I built Raptor. That engine used RocksDB to store the content on disk.

Kero: Two of my friends from the lab were following my progress, and they approached me about creating a company around this search engine I had built. That sounded good to me, because I much preferred working on my search engine to the things I was doing at my job. So, the three of us started a business. We really started with the notion of being “intrapreneurs” within Veepee, but we couldn’t find a clear path to do that. That was taking too long, so we went out on our own with the idea that we would find funding somewhere. So, we went back to the 42 School and used it as our offices.

Kero: Our big break came when Louis Vuitton decided to sponsor a hackathon called something like “Unlocking Search Power.” The goal of the hackathon was to improve the way sellers in Louis Vuitton shops were able to answer questions from customers. For example, if a customer showed up with a vintage bag, they wanted to be able to find out about the history of that bag. We participated in the hackathon and created a really complete solution for them. We created a server for the search engine and an iOS app to show the results. Surprisingly, we won first place! We even beat out teams of engineers from Algolia and Elasticsearch. As a reward, we were given offices and a contract where we worked with them to improve the solution we created so that it could be used in production. It was a dream really.

Kero: This was also where our open source model came about. It was written into the contract with Louis Vuitton that the entirety of the search engine would be open source. So, it’s been open source ever since then.

Drew: Okay, there was a lot in there I wanted to ask about. So, all of your initial funding came from Louis Vuitton?

Kero: Yes, so it wasn’t venture capital funding or anything like that. They were our first customer, and we were able to fund the company off of our invoices from them. As the founders, we were still getting some money that the government gives to help people find jobs. So, between those two sources of income we were able to get by.

Drew: I see. Where did the Meilisearch name come from, because you changed it from Raptor, right?

Kero: Yeah, so the company is named Meili, which is the name of the norse god of forgetfulness. So, like discord is the name of what the company Discord is fixing. At Meili, we’re fixing forgetfulness. It also happens to mean beautiful in Chinese, but we weren’t aware of that when we picked the name!

Drew: Beautiful works too! So, in the Louis Vuitton competition, you beat out Algolia and Elasticsearch. Is that because you chose Rust, or were there other things you did that made the difference?

Kero: Well, Algolia is expensive. I also think the Algolia team was a little too sure of themselves, because Algolia is used by probably all of the other LVMH companies. I don’t think Louis Vuitton liked that. They’re the biggest company in the LVMH group, and they view themselves as setting the standard for the whole company. So, for them, they wanted it to be clear that they drive the company and they would make an independent decision. I don’t know this for sure. This is just what I’m guessing.

Kero: Technically, Algolia didn’t index all of the data we were given. We did. I think that was the key reason our solution was better technically.

Drew: Do you know why Louis Vuitton decided to run that competition in the first place? It seems like kind of an unusual way to solve a problem.

Kero: They love competition. And, it was a really smart way for them to show off the technical side of Louis Vuitton. It got people talking about them as a company with a strong technical side. I may not have made this clear, but it’s worth noting that the competition was hosted within the 42 School.

Drew: Oh, so it was affiliated with the school?

Kero: Yeah, so we were very lucky to be there at just the right time.

Drew: So, you mentioned earlier that when you were at Veepee, you disagreed with the choice to use golang for the search engine because of the garbage collection. Is that because you would have searches that get interrupted by the garbage collector?

Kero: Exactly. We measured the response time for different queries on a graph, and it was basically a flat graph. But, sometimes you’d have this huge, long-lasting spike. It wouldn’t affect just one request. Often it was like three or four requests that would be impacted. Even then, my search engine didn’t exhibit this behavior. My implementation was also very consistent with its memory usage. With the Go engine, you could see memory usage going way up and down over time.

Drew: The other thing I wanted to ask about was the decision to go open source. It sounds like that was part of the contract with Louis Vuitton. Can you talk more about that decision and how it affected the business?

Kero: So, I was and still am a very big open source contributor. Raptor was actually open source from the beginning, even before Louis Vuitton. And, I was using and contributing to several open source libraries as part of that process building the initial engine. So, we decided to go open source because the Rust community was just getting going, and there weren't a lot of Rust developers. So, we hoped to be able to take advantage of a larger pool of talent by keeping it open source. And, it’s amazing how much we’ve been able to benefit from open source contributions.

Kero: Are you familiar with Sled?

Drew: No.

Kero: Sled is an open source key value store written in Rust. We tried to use it to replace RocksDB. I contributed to the project, and we even gave money. In the end, it didn’t work out for us. But, I think our work with Sled shows the power of open source. We contributed to their project, and they actually contributed to ours. So, you give and you get.

Kero: Another example is Actix. We use Actix for our http server, and often the people from Actix will actually update Actix on Meilisearch for us. They just do a PR and bump the version to the latest for free. It’s very kind of them!

Kero: We also got a huge performance boost with our roaring bitmap for free from a guy who just decided to help us on that. We would have paid for that work! It’s inspiring how many people will contribute to software they like just for the fun of making something great.

Drew: That is very cool. It’s such an amazing aspect of the software industry.

Kero: I should say that our business really hasn’t been impacted by the open source decision. Getting the best out of the search engine requires knowing it really well. And, the pieces that aren’t open source end up being quite complicated as well. That’s the cloud infrastructure mostly. We’ve done a lot of work to make sure it’s running on the best hardware. Kubernetes is hard. Stuff like that gets complex. Beyond that, creating a website where you can request an instance, manage payments, etc. is complicated as well.

Drew: That makes sense. So, it’s kind of the open core model?

Kero: Yeah.

Drew: Which of your competitors are open source?

Kero: Solr is open source and Elasticsearch is open source. Algolia is not. But, most of the search engines are open source.

Drew: Interesting that Algolia is not. I heard someone the other day comparing OpenAI to Meta’s open source AI efforts. They were basically saying that when you’re the first in a market you want to be closed source but then when you’re coming from behind you use open source as a strategy to catch up.

Kero: Yeah, that’s an interesting take.

Drew: I assume you wouldn’t have just known how to build the engine. It’s not exactly something they teach you in school necessarily. How did you figure that out?

Kero: I read Algolia’s blog posts very closely to get an idea of how to rank documents. From there, you can deduce a lot. Once you know how you want to rank things, you just have to build everything to make the ranking possible. But, what gets complicated is performance. Search can be resource intensive on all fronts. You have to carefully optimize it to be fast and not use too much memory or storage. That’s where most of the work is now.

Drew: So, it’s basically just a matter of figuring out ranking and then everything else from there is optimization.

Kero: Exactly.

Drew: I’ve played with some of your demos, and they are very fast.

Kero: Thank you. They can always be faster.

Drew: Of course! (laughing) So, Meilisearch is basically all Rust, is that right?

Kero: Yes, but we do have a huge amount of dependencies, some of which aren’t available in Rust. For example, we use the zstd compression library from Facebook. That’s C++. Our biggest dependency is LMDB, which is our key value store. We made that switch because RocksDB was using too many resources. And, that is a C library. Everything else is in Rust.

Drew: I read that you think Rust is one of your biggest competitive advantages. This caught my attention, because I saw a post the other day on the Rust subreddit where someone was asking why companies say that their product is built in Rust as if it's a feature. Why do you view Rust as a competitive advantage?

Kero: I mean, the number of hours we’ve spent debugging a segfault since the beginning of Meilisearch is about ten, and it was only due to a C library being used in a bad way. So, Rust makes us faster in the long run because you basically can’t compile a program that will crash at runtime. There’s a whole class of bugs that we don’t have to deal with. We also use a lot of parallelism. Parallelism is significantly easier and safer in Rust as well. Also, the U.S. government has basically stated that Rust should be used instead of C or C++.

Drew: I saw that statement from the government that you’re referencing. It seems like they’re moving toward saying that future software must be written in memory safe languages. If something like that happens, it will be a crazy change. That really will be a competitive advantage.

Kero: Yeah, all of the C++ libraries would be effectively obsolete for future use. With that said, right now you can’t find a lot of things in Rust. For example, you can’t find a postgres or mysql equivalent in Rust. It will still take time.

Drew: Have there been any positive or negative surprises you’ve encountered while building in Rust?

Kero: I knew you would ask something like this and didn’t have many answers myself, but I was able to source some answers from my team. So, one of the things is that we’re really pleased with the capability of the average Rust engineers. Rust developers seem to be more capable than average.

Drew: Do you have any idea why that is?

Kero: I don’t really know. I have a theory that because Rust developers are forced to fight the compiler, it weeds out anyone that doesn’t truly want to write the most correct code possible. I remember being really discouraged after my first month in Rust, because I was having such a hard time getting my programs to compile. I even went back to C++ for a while, but I quickly decided that life in Rust was better. So, I forced myself to figure out Rust. That’s where my two years of rewriting school projects in Rust came in. I will say, there’s also a downside to all of this. Often, it takes us longer to release things because our engineers always want it to be perfect. But, in the end you have a really robust program.

Drew: What type of projects were you doing in school?

Kero: I created a ray tracer that was replicated across a cluster, I also built a clone of a game that was totally parallelized. Mostly it was a lot of very basic, puzzle-solving sort of programs.

Drew: So what other surprises did you notice when you were building?

Kero: I’ll give you the positives, and then I do have a few negatives.

Drew: Okay.

Kero: I was surprised to find that the upgrade process is very smooth when you update dependencies or even the compiler itself. You still get error messages, but the compiler basically walks you through what you need to do to get things working again.

Kero: I’m also very impressed by the documentation of the Rust standard library and the documentation that’s available for crates.

Drew: I’ve definitely noticed the “just follow the compiler” type of debugging. It’s really nice.

Kero: Yeah, it can be kind of monotonous, and maybe one day an AI will be able to resolve those sorts of errors. But, it’s a straightforward upgrading process. I’m very happy with Rust’s error messages, and a lot of that is due to the type system. Even C++, which is a typed language, isn’t as strong as Rust because of the automatic conversion between types that it allows.

Drew: Alright, what are the negatives?

Kero: There are some basic crates that are missing. For example, we had a hard time finding a good raft library that was simple to use and worked well. There really isn’t a great library for that. For a long time, we were using the beta version of Actix. It’s stable now. Also, we had to modify the roaring bitmap library. I had to modify it a lot to improve the performance, while the C versions did what we needed already. I’m the main contributor to that crate, and I know for a fact right now that the C version is still faster. But, it’s coming along.

Drew: There’s been a lot of talk about the idea of LLMs fundamentally changing search. How has that changed the way you think about Meilisearch?

Kero: We think AI is really important, and we’re absolutely embracing it. We’re releasing a lot of AI-related features that make us compatible with OpenAI and other big AI players. Ultimately, we want to provide the best search engine on the market, and we think AI is part of that. We’ve already done a lot with vector or semantic search, and we’re also working a lot on hybrid search. Hybrid search is a combination of the traditional keyword search and vector search. Our hybrid search implementation is showing really great results. Hybrid search is especially great at surfacing documents that might be relevant to answering a question. Keyword search doesn’t do well with questions, but hybrid search does.

Drew: Have you seen it impact your work in other ways?

Kero: In terms of writing code, we don’t write Rust with any sort of AI assistance. But, sometimes I have to do things that I don’t know well, like React. In that case, I will get some help from ChatGPT. And, by the way, this is all on Zed.

Drew: Oh, you use Zed?

Kero: Yeah, it’s very good! So, when I do things I don’t know well, I use some AI.

Drew: Can you talk a little bit about the culture and work style at Meilisearch?

Kero: So, we are fully remote. Our primary language of business is English, but sometimes we speak French in meetings if everyone in the meeting speaks French. We also work a four day week. We take Fridays off. In general, I would say we’re very customer-oriented. We spend a lot of time trying to anticipate the problems our customers might have.

Drew: Is the four day work week working well for you?

Kero: Yeah it does. I find that at most companies most people don’t work on Friday anyway. That’s mainly why we decided to skip it. I find that most Fridays I decide to work anyway. It’s fun for me to just code without any meetings or other distractions. Often, I spend the time optimizing the engine. Usually, I have an idea in the back of my head that’s outside of the normal sprints that I just need a chunk of time to work on. Recently, I was able to make the engine filtering system six times faster with one of those optimizations.

Drew: That makes sense. It’s a time for you to have deep work. You said the primary language is English. Does that mean you have team members all over the world?

Kero: Yeah, we have two people in Prague, one in Ireland, one in Brazil, one in Argentina, one in Spain, and a lot of people in France. I think that’s probably everyone. We also want to hire people in the U.S., but mostly sales people and such. We don’t have people further East because it gets hard to work together with conflicting timezones.

Drew: So, what does the future look like for you and Meilisearch?

Kero: So, I’m really focused on releasing benchmarks against BM25, BM42 (laughing), Algolia, and Elasticsearch. I think that will really show people how impressive Meilisearch is. We want to become the default search engine in the future. I think the most important thing to achieve that going forward will be using more AI to answer semantic-type requests. So, that’s the vision for Meilisearch. For me, Meilisearch is my baby so I’ll be driven by the life of my child. (laughing) I’d like to do a better job myself of communicating about Meilisearch. I’d like to grow my followers on X and do more presentations and things.

Drew: So, if Meilisearch is your baby, would you ever sell it? Or, will you run this business forever?

Kero: When you put it like that, it makes it hard to say I would sell it. I suppose it depends on the acquirer. We’re very happy about the current trajectory. So, maybe one day we would be bought by another company. I would just want to make sure that it's a company that has similar values to Meili. We try to be very ethical, so I would want that to continue after a sale.

Drew: That makes sense. Well thank you so much for talking!

Kero: Thank you!