IBM Quantum: Rust is Real, but Quantum Advantage is Not (Yet)

Drew: I feel like quantum computing is one of those things that a lot of people don't know much about, but there are also a lot of opinions swirling around about its potential effects, whether it's going to come online in a big way soon, and all of these things. So, I thought a fun or interesting place to start would just be to give you an opportunity to dispel any myths that you tend to hear about quantum.

Matthew: It's a fun one because whenever I give a talk on an intro to quantum computing, I always start with this. The first thing I always show is that it's not science fiction. I show a picture from an anime I like where they had a quantum computer that was a big plot point. But the other thing is, it's not going to magically break everyone's encryption.

Matthew: There's been a theoretical algorithm that could factor prime numbers, which is the basis of RSA encryption. But the quantum computers needed to do that are theoretical and well outside the scale that we can build them today. So, your encryption is safe for the time being, minus the case where people harvest encrypted data and then plan to decrypt it in the far future.

Matthew: The other misconception that I see a lot is that people think quantum computers are special kinds of supercomputers, and that they're going to replace the computers we all use every day. That's not really what they're being built for. They're being built to solve problems that are intractable on our current computers.

Matthew: It's in the NP-hard kind of space, where you need exponential resources to try to solve them. All of our methods for solving these problems on normal computers are some kind of approximation, some kind of numerical approximation. What quantum computers are being built for is to solve these other kinds of problems that we don't have a good answer for on normal computers.

Matthew: I might slip and say the word classical computer. I don't like it. It's a physics thing; there's quantum physics and classical physics. So, they call them quantum computers and classical computers. I’m not a big fan of that term, but I'm probably going to slip and say it.

Drew: Okay, yeah, I very well may say that as well. What do you prefer?

Matthew: Normal computers. I don't know. “Classical computers” makes them sound old, but they're still new, and it's what I do every day.

Drew: Right, they're not old. Anything else you wanted to say? I kind of cut you off.

Matthew: No, no, I think I could go on for quite some time on all these misconceptions about quantum computing. The more you learn, the more you dispel some of them, but then pick up other ones. Another one I hear is… Well, it's not worth getting into.

Drew: Okay, well, if you want to get into it later, you can bring it up.

Drew: I'll go ahead and dispel another myth, one that I held. I thought that quantum wasn't really being used in a big way right now. I was doing some research on IBM Quantum and was quite surprised to find that it's a revenue-generating division of the company. I think I saw the number was over a billion in cumulative revenue. That kind of broke my frame. Where does that revenue come from? What are the use cases?

Matthew: The product offerings that IBM Quantum has are what we call quantum computing services. We build quantum computers and then offer them to clients to run whatever workload they want on them.

Matthew: For the ones that I think are the hottest topics right now, there are two categories. One is simulating nature. I think there is some Richard Feynman quote about needing to use a quantum computer to simulate nature, because nature is quantum. That is an area of active research. At the end of the day, all of these applications are research. No one is using it right now for a business-critical system that I’m aware of. It's not like credit card transactions are going through a quantum computer for some processing. They are all research applications.

Matthew: Simulating nature is like chemistry simulation. People are looking at it in all sorts of areas like battery development and drug discovery. The other area is optimization problems, such as numerical optimization for portfolios in finance or max-cut for graphs. These are the normal mathematical optimization problems that are a big part of numeric computing.

Matthew: I would say the client base is pretty wide at this point because everyone is seeing the potential in quantum and trying to get in on the ground floor before they're left behind. I think there was a very interesting result that came out of HSBC recently where they claimed that if they used a quantum computer for some financial application they would have gotten 34% better results than the best techniques they had before.

Matthew: The way our corporate marketing frames it is that there is a period before what we call quantum advantage, which is the point where a quantum computer is the best tool for doing some task in computing. Everyone is trying to find evidence that there is quantum advantage, or some application where you can use a quantum computer to do it better than any other tool.

Matthew: We are in this regime right now of what we call quantum utility, where a quantum computer is the best tool for doing quantum computing. This is counterintuitive, but in the early days, you could simulate a quantum computer better than an actual quantum computer. Now, we are at the point where quantum computers are better at being quantum computers than simulators. So, we are trying to find an application where there is a real advantage to using one every day.

Drew: That's very interesting. So, to go from simulators being better than the real thing to the quantum computers themselves being better is a big landmark. Maybe not a big landmark for real-world use, but it's a definite step, right?

Matthew: Yeah, that's definitely a big step. And it kind of marked this whole new excitement in the field. You saw a lot of excitement in people trying to push what they could actually do on the hardware.

Drew: The HSBC case you mentioned—you said it was about 34% better. What does "better" mean?

Matthew: It was prediction accuracy, if I remember correctly. They published a blog post and a paper about it. They were doing some kind of trade speculation and wanted to estimate something. They were using a computer to help make a decision on whether to do a trade or not. If they used the quantum algorithm they developed, they claimed they would get 34% better results.

Drew: Okay, that makes sense. And then is there anything exciting that's come out of the quantum chemistry side that you know of?

Matthew: Not practical applications yet, but the work I'm aware of focuses on trying to beat the approximate state-of-the-art classical algorithms. This involves simulating the ground state energy of a molecule on a supercomputing cluster and then trying to do the same on a quantum computer to achieve a better, more accurate result.

Matthew: Last year, we published a paper with RIKEN, the supercomputing center based out of Japan, where we beat many classical methods for simulating certain molecules using a quantum computer. I believe it was an iron-sulfide molecule, and we computed its ground-state energy. I'm not a chemist, so I don't understand the details.

Drew: So, I find it really hard to get a sense of the potential impact of quantum computing, and especially how quickly it might arrive. From your better perspective, is there anything you could say to flesh that out? Are there certain bottlenecks you pay a lot of attention to that give a sense of how things are progressing?

Matthew: For the macro-scale question, I'd say the impact will be very specific at first. There will be initial applications where a quantum advantage is determined. In those fields, people will be really excited, and it will become the hot topic—like in chemistry or finance. Other industries will go through a hype curve, just as with any new, promising technology.

Matthew: However, the practical impact will be in very specific fields initially. Longer-term, I’m not sure. There is that looming threat of Shor's algorithm that I mentioned earlier, which is the ability of quantum computing to break RSA encryption. That's what everyone is worried about decades in the future, and that's the one thing that will impact everyone.

Matthew: As for metrics of progress, there are a lot of benchmarks out there to try and measure how good a quantum computer is. None of the ones I've seen really capture the practical side. They are all targeted toward saying, "My quantum computer is the best because it can do this," whether that's in terms of scale, speed, or quality of results. From a practical perspective, if you're not in the industry, those metrics don't mean much to you. For example, "I can do a computation of this size in the quantum domain on this quantum computer" doesn't tell you very much if you don't know how quantum computation works.

Matthew: So, from the perspective of measuring progress, you can look at those benchmarks, and they always go up. Every year, they increase, whether it’s the number of qubits—which shows the scale of the computation you can do—or metrics like Quantum Volume or CLOPs (which is one we use at IBM that measures the speed of execution). As long as those are increasing, there is progress in the hardware. But, what I think is more interesting for the software engineer or outside observer is the algorithms. Unfortunately, there are fewer good benchmarks to measure progress in that field.

Drew: Yeah, that makes sense. I imagine the benchmarks would have to be very specific to the application the algorithm is targeting.

Matthew: Yeah, it's hard. You could come up with a benchmark for a very specific algorithm and run that on a quantum system, but then you're benchmarking that particular algorithm on that system. It becomes a chicken-and-egg problem: if you don't have an interesting application, how do you even write the benchmark?

Drew: It's so fascinating to learn about this field at this point in time. This next question is a little less about quantum generally and more about IBM specifically. When I think about IBM, I think about mainframes, storage, and consulting. How did IBM get involved in quantum? That seems kind of out of left field.

Matthew: IBM has been involved in quantum computing for a very long time, since at least the 1980s, if not earlier. Some of the early researchers in quantum computing were actually at IBM.

Matthew: IBM has always maintained a very strong research arm. As a computer company, we are always interested in researching things that lead to computing, whether that's the theory of information or physics. So when quantum computing was initially proposed as a theory, there were IBM researchers interested in it.

Matthew: Since those early days, there has been a team at IBM conducting research in quantum computing. I used to have a timeline for introductory talks that showed a bunch of milestones of IBM's involvement in quantum computing, and it goes back really far.

Matthew: I'd say a significant milestone was when the team began building physical quantum computers. The biggest milestone was around 2016 when IBM made one of these computers available on the cloud, opening it up to everyone. They had been developing and iteratively improving that technology for quite some time beforehand.

Drew: That's pretty interesting. IBM has basically been involved since the beginning then. I feel like it's something that could kind of come out of nowhere and make IBM grow really explosively if people start finding applications that truly work.

Matthew: That's the company's bet at this point. They're investing very heavily and pinning a lot of hope on it. If you look back at where the technology was in 2016 (before my time; I joined IBM Quantum in 2018 ), that first system was very basic compared to the scale of the systems we have available now.

Matthew: That first system was only five qubits, really noisy, and had a lot of errors. Now, I believe our biggest one in production is 156 qubits, and the error rates are orders of magnitude better.

Drew: On that note, since you're talking about how much the team has improved, have there been any big milestones for the IBM Quantum team lately?

Matthew: I'd say I've mentioned two of the bigger recent milestones. The first was the HSBC result , which was a big deal from a business perspective, as a client claimed quantum computing would have a real impact on their business. The second was the chemistry simulation from last year , which showed a new technique using quantum computers in conjunction with a High-Performance Computing (HPC) cluster. The other big milestone, which is only two or three years old, is what we call quantum utility. That's the point where a quantum computer is better at being a quantum computer than a classical simulator is.

Drew: It's interesting that you mentioned those because they're not the highly theoretical things you were talking about before that are hard to track. Those are pretty tangible.

Matthew: At least for me. That might be my bias as an engineer speaking, but for something to be important it has to be practical.

Drew: Right, right. Maybe as an engineer versus a scientist, that's what's interesting to you. You are a software engineer , which is really interesting, rather than being a physicist or someone else involved in this field. As I was preparing for this interview, I wondered what it actually looks like to write software for quantum computers. It can't be similar to what we have now. With normal computers (or classical, as I almost said), we have operating systems and are so abstracted away from the hardware. I imagine that's not the case with quantum. What does it look like?

Matthew: It's much lower level at this point. The unit of computation on a quantum computer is called the quantum circuit. You run individual operations called gates on each individual qubit or sets of qubits to change the qubit's state. Then, you measure it out to get a bit string, as each measurement has a binary result.

Matthew: You typically run the quantum circuit many times and then post-process the data to get a result. It's a much lower level of programming than what you'd be used to. There are proposals for quantum programming languages with higher levels of abstraction, but in my opinion they're premature at this stage of the hardware. To get real results, you still have to program at a low level. I often think back to computer history and manually programming with DIP switches; this is the digital equivalent of that.

Drew: Right, yeah, that's so interesting. That could be kind of fun. So are there sort of pre-made circuits that you can take and drop in to build on top of, or is it literally like switch by switch every time?

Matthew: No, no. The way the research is done is that you build a library of circuits—like in the package I maintain, Qiskit—that are designed to prepare quantum states in certain ways. You provide parameters for what the circuit should look like, and then you execute that as part of a larger workflow.

Matthew: This is the typical design for these algorithms. You define a circuit of interest or a specific way to prepare a quantum state on the quantum computer. You then sample it stochastically to measure it, get those bit strings out, and process the results.

Matthew: You write a function to build up and submit that circuit once, and then you vary the parameters. You're not sitting there manually writing out every instruction on every qubit. You write software to do that for you, which makes it reproducible and easier to use.

Drew: Okay. So, yeah, you mentioned it—is it KissKit or QuizKit? How do you say that?

Matthew: I think KISSKIT is the official pronunciation. There was a debate early in the project's history; people even made stickers about "KISSKIT or KWISKIT".

Drew: So, an internal feud was had! Okay, KissKit. You’re the architect of Qiskit, so tell me about it. A lot of our audience probably won't have any idea, so tell me what it is and how it works.

Matthew: Qiskit is an open source library, an SDK for programming quantum computers. I like to describe it as a combination of libc and gcc.

Matthew: It gives you an in-memory software representation of a quantum circuit and the operations on it, allowing you to build those up in software. It also has a compiler framework for taking those quantum circuits and targeting them for any hardware, whether it's IBM's or another vendor's quantum computer.

Matthew: Qiskit was originally built in 2016 to have a Python API to interface with the quantum computers IBM first put on the cloud. It has since evolved to be more general purpose as it gained popularity.

Matthew: One of the biggest changes is that it’s moved from being pure Python—now, a lot of it is written in Rust. I estimate about 40% of the lines of code are Rust. The core data model and everything we're doing in Qiskit is based in Rust, with a Python API and also now a C API wrapped around it.

Drew: Okay, so that's very interesting. You’re basically interfacing from Python to the Rust core. That's cool. When I reached out, I wasn't sure how big of a role Rust plays, but it sounds like it's pretty huge.

Matthew: Yes, I started using Rust for Qiskit, but not in Qiskit itself initially. It was in a project I started called RustWorkX. It was originally called RetWorkX, which was my attempt to make a faster version of the popular Python graph theory package, NetworkX, because NetworkX was not fast enough for what we needed in Qiskit.

Matthew: So, we began using Rust for that separate library, which was a graph theory library for Python written in Rust for performance. It was very successful for Qiskit and has gained traction outside of Qiskit in both quantum computing and other fields that use graph theory or network science.

Matthew: Now, we've incorporated Rust into Qiskit itself, and it has been very successful for both performance and stability. This pattern has also translated to a lot of the internal software we write at IBM Quantum, which is often being written in Rust.

Matthew: For example, when you submit a quantum circuit to be executed, a lot of the lower-level processing that takes the job and executes it on the quantum computer is an increasing amount being written in Rust at this point.

Drew: As a Rust person, that's just really exciting. I just had no idea that it was so significant. So, let me frame this question: I've been talking to a lot of embedded programming and robotics people lately, and it's becoming clear that Rust is naturally suited to those use cases. Is there something similar with quantum? Are there specific reasons why Rust works so well there?

Matthew: I'm not actually sure there is an intrinsic advantage specific to quantum computing. The advantage of Rust in the quantum field is primarily the general software engineering advantages.

Matthew: For example, you don't have to worry about memory safety when writing the code. You find surprise bugs while you're writing the code, not in production. It offers ease of maintenance and a mix of high-level language features while providing low-level language performance. All of those general advantages for writing good, easy-to-maintain software in production were the reason we started using Rust.

Matthew: I will highlight one thing: I really like Rust for an open source library like Qiskit because the compiler is checking so much for you, which takes the edge off of code review or maintaining code from random people. That was a huge advantage.

Matthew: Quantum computing attracts a lot of interest, and we get contributors with lots of varying skill levels. The compiler essentially checks so much for you that you don't have to worry about things like dangling pointers. If the code compiles, you have some assurance and can concentrate on the logic, which is really valuable.

Drew: Yeah, I've never heard that put together—the open source benefits—but that's so true. I hear people talk about building up business logic in the Rust type system within a company, knowing that when someone does a refactor, the compiler will walk them through everything that needs to be changed and ensure a certain level of correctness. But that benefit would be so amplified in an open source context.

Drew: That's really cool. That piqued my curiosity: What is the scale of the participation in the Qiskit project? Is it pretty huge?

Matthew: I've worked on larger open source projects in the past, but for a field like quantum computing, I think it’s pretty big. Qiskit has had about 700 unique individual contributors over the history of the project. I'd say the bulk of the work is still driven by IBM, but there is definitely a fair number of individual contributors who are not from IBM. Seven hundred unique contributors is a lot, especially for a lot of open source projects, though it's not on the Linux kernel scale.

Drew: Yeah, yeah. That's a lot. And, it was cool you mentioned that you get a lot of different skill levels. Do people tend to be really productive if they're fairly novice, or is it more just something that they're poking around the edges of?

Matthew: I mean, there have definitely been examples where people show up with little experience and they grow. But, to be honest, that's kind of the exception rather than the rule. A lot of novice users show up and get frustrated that it's not passing CI. It's hard to overcome that barrier unless you have a mentor who is walking you through everything to get started.

Drew: Yeah, that makes sense. So, one other question on the Rust note. Do you find that the Rust ecosystem kind of provides everything that you need in the quantum computing realm, or are there some things left to be desired to make your work easier?

Matthew: I'd say from a pure language perspective, we are very happy with what Rust gives us. The area in the ecosystem that I think holds us back a little bit is linear algebra libraries.

Matthew: The space there is fractured. There is ndarray, an n-dimensional array library that doesn't really do linear algebra unless you link against BLAS and LAPACK, which are standard C or Fortran libraries. Then there's n-algebra, which has some linear algebra functionality but also links against LAPACK for more sophisticated things.

Matthew: The one we've been using more recently is Faer. It's a smaller project, primarily a single maintainer who does a great job, and it is pretty performant and all Rust. But it's a little immature as a library; it's still new and evolving quickly, so it can be hard to keep up with changes. Since we do a lot of linear algebra in quantum computing—because linear algebra dictates what these quantum operations are doing—that has been a pain point at times.

Matthew: Graph theory is the next thing we do a lot of in Qiskit. There is one big library for that, petgraph, which has gone through ebbs and flows of being actively maintained, which has been an issue at times. That's partly why I created RustWorkX, which is now a standalone library that wraps petgraph and extends it with extra functionality to try to fill that gap.

Drew: Maybe we can kind of put the bat signal out. Sounds like there's some additional work to be done there.

Matthew: Yeah.

Drew: Yeah, that's good to know, because those two areas specifically affect a lot more than just quantum, so it's got to be a pretty big need.

Matthew: Yeah, I would say so. nalgebra is pretty mature at this point, and I think it solves a lot of use cases. It seems to have some focus on computer graphics, which makes sense since Rust is popular with people developing game engines. For that functionality, I think it's better. Luckily, there's some overlap in the math we do for quantum computing, but not all of it.

Matthew: My biggest complaint about linking against LAPACK is that I don't want to complicate the build system with needing an external system-installed library or building it from source. Our users tend to come from Python, and they just want a package to install. Rust is great for that because everything statically links, and Cargo makes it very nice. But the second you say you need a giant C++ or Fortran package, those advantages go away. However, if you are writing a heavy linear algebra application, maybe that's not as much of a concern.

Drew: So, we kind of started talking earlier about people contributing to Qiskit and novices getting into the ecosystem. I'm sure there are quite a few people out there who are curious about quantum. How does one end up getting into quantum computing? Do you have to have a pretty strong physics foundation, or can you come in as a pure programmer?

Matthew: So, I came in as a pure programmer. My college background is in computer engineering, and I specialized in chip design, specifically VLSI, because I wanted to make microprocessors. I couldn't find a job in that without a master's degree, so I went into open source software as my career.

Matthew: I worked on different projects, including the Linux kernel, though I was not very successful. I then spent a lot of time working on open source cloud computing tooling, primarily the OpenStack project, which was all Python.

Matthew: I joined IBM Quantum on a whim. I was planning to quit IBM but heard about an opportunity in IBM Research to do open source software development for quantum computing, and they were looking for people with software experience. I had no background in quantum computing or physics. I had some basic physics from electrical engineering in college, but my software skills were strong and fresh.

Matthew: If people are interested in working in quantum computing, it is a field of diverse expertise. Yes, you need people who are experts in quantum physics, but you also need software people, people who can design hardware, and people who can do mechanical systems for cooling. You need a real breadth of different expertise, and no single person can do all of it.

Matthew: You are not going to find one candidate with a PhD in quantum physics, 30 years of experience doing Rust programming, and who also builds cryogenic fridges in their backyard. People like that don't exist. Everyone has a specialty, and it's combining all those skills that lets us make progress.

Drew: It sounds like it's a field where you're doing a lot of learning on the job.

Matthew: Yeah, definitely. Constantly learning. That's part of why I've stayed for so long. It's always new, I'm always learning something new. Just when I think I have a grasp on something, there's a whole other dimension that I didn't know about. I've also learned a lot of new types of math that I didn't know existed before.

Drew: That's always interesting and potentially daunting. But I think that sounds fun, to be able to be in a field where you can just learn so much. You mentioned that you've stayed because you continue to learn. You've been there for what, like seven years now?

Matthew: Yeah, I joined in the second half of 2018. I guess it's the longest job I've ever had.

Drew: There you go. It sounds like it's working well. Given that context that you have working in the organization for a good chunk of time, what are the qualities you see that tend to make people really successful, or maybe really not successful, working in this space?

Matthew: I think the biggest one is that willingness to learn and also to say that you don't understand and ask to learn. There are so many varied aspects to quantum computing—building the hardware, or the software for them—that you can't know everything, and people have different expertise. The people who tend to be really successful are those who know when they don't know something and are willing to ask and learn, and then put that learning into action. The people I've seen who aren't successful aren't willing to do that, and they can't keep up.

Drew: Right. So the scale of the challenges you're facing are such that you really have to have intellectual humility to just keep asking questions. Beyond what you just described, is there anything noteworthy you would say about the culture at IBM Quantum or maybe just the way the people there are?

Matthew: There was a phrase that I mocked internally at the time, which I think I'm going to get backwards—it was either "think like a startup, act like IBM" or "act like a startup, think like IBM". I've worked in several different divisions in the company, and the quantum division is unique in many ways. It often feels like a smaller company. It tends to be a pretty agile place where people move around a lot, learn different things, and things are constantly changing. It's an interesting mix where they try to keep everyone thinking fresh and innovating constantly instead of following a rigid process.

Drew: So, more like a startup a little bit. That's cool. I think from a recruiting standpoint, that's fairly convincing for a lot of people to hear. For people worried about bureaucracy or whatever, to hear that is probably really positive.

Drew: We talked about your path to starting a career in quantum. Would you have advice for people who want to do that themselves? What are good ways to crack the door open and start the process?

Matthew: I'd say the first thing is don't be afraid if you feel like you don't know everything. Take a chance and apply, because your skills might be useful, and you don't even realize it. There is always a need for good software engineers, and every business is a software business at this point. The other thing is, because quantum is so new and a different computing paradigm—where you're not writing code or maintaining information the same way—there are a lot of free resources out there to learn about quantum information theory and quantum computing. Take advantage of those to get some background information; a little bit goes a really long way. If you want a career in quantum computing, taking advantage of all the free resources available to teach people is the best way. I did it a little backward—I worked out the transfer and then started taking advantage of the free resources immediately after they accepted me. But it helped a lot because there are so many new things to learn.

Drew: Are there any of those resources that you would shout out specifically?

Matthew: I have my little sphere that I know, and it tends to be pretty focused on the Qiskit and IBM Quantum story. There might be other resources out there, but the best one for me is the Qiskit learning course that we do on the IBM Quantum platform. That tends to be pretty well done, in my experience. The resources I used, a lot of them don't exist anymore—they're pretty dated. I think the learning platform was an open-source textbook, but that wasn't the highest quality. I think you can still find it because it's open source, but it's not maintained. But I think that the IBM Quantum Learning Platform has a lot of good resources, and I'm sure there are other ones out there, too.

Drew: Well, I think I asked everything I wanted to ask, but I always like to ask if there's anything else you wish we'd been able to discuss, or something that maybe popped into your head, or something you want to leave our audience with.

Matthew: No, I can't think of anything, to be honest. We've been talking for a while and covered quite a few different areas.

Drew: I think we can end it there. Thank you so much, Matt.

Matthew: Thanks. It was great talking to you.