Podcast: An AI-first data scientist on its limits and potential

MOLLY WOOD: Novak also has a deep background in data science, and he shares unique insights for leaders on how AI is transforming what they can do with all of their data, and how to get at the untapped value it can hold. Here's my conversation with Kevin.

MOLLY WOOD: Kevin, thanks so much for coming on WorkLab.

KEVIN NOVAK: Thank you for having me.

MOLLY WOOD: Okay, you are a self-described data nerd. I'm excited to talk about the actual beating heart of all things AI. Can you tell us a little bit about your career journey as a data nerd?

KEVIN NOVAK: Yeah, sure. Happy to. So I always tell everyone I've been a data nerd basically my whole life, it's just that these days I get paid for it. Had an academic career, the beginnings of one in nuclear physics, and transitioned into data science in the late 2000s. Jumped into startups. I was a very early employee at Uber, was chief data officer of a couple other startups, had a couple of failed companies that I tried to start, and then got really interested in the business of investing in data companies. That was initially me doing an angel investing, sort of side hustle nights and weekends, while I was building companies, which ultimately culminated in us launching Rackhouse in the fall of 2020.

MOLLY WOOD: So let's talk about AI and, specifically, tell me about what you saw transforming the field of data science, especially large language model AI. Like, what is possible now that wasn't feasible before that you think is super investable?

KEVIN NOVAK: I think transformers and large language models and all of that, that's gone from a "wow, this is impossible" to almost commoditized at this point, which is interesting from an investor and, I'm sure to the audience, because a significant amount of the day-to-day jobs in a modern workplace, and especially in the knowledge business, the deliverable is sort of generated text, right? It's like we do this thing and then you get an email or you get a PDF or you get a report, right? Like, that is the deliverable, and humans were always the primary driving force for getting a bunch of these projects and those kinds of deliverables across the finish line. Data would advise on strategy and then humans would execute. I think that that balance is starting to shift.

MOLLY WOOD: I've also heard you say the word transformers. To also mean LLMs?

KEVIN NOVAK: Yeah, sure. So there's a couple related concepts, right? So, a transformer you can think of as the fundamental building block of a large language model. And then things like-

MOLLY WOOD: Copilot.

KEVIN NOVAK: Right. These are specific brands, specifically trained models of a large language model.

MOLLY WOOD: Just want to make sure we get all the definitions out of the way up front. Okay, so let's look at the startup landscape. I am of the opinion that when things are very new like this, there's opportunity, but also opportunity to lose a lot of money. How are you using your expertise to evaluate the landscape around AI startups? Because it seems like there's one born every minute.

KEVIN NOVAK: Very much the case. I would say that 1) this is an absolutely bonkers time to be investing in building an AI. We're definitely in the middle of a gold rush of some form or another. But I always think it's interesting, because one of the reasons why I started Rackhouse was that I kind of felt like, in my entire career in data science and AI, the peaks and valleys felt really big. I remember right when I started at Uber, within the first year-I think it came out of HBR, but there was this famous article about, like, data scientist is the sexiest job of the 21st century and everybody needs to have it. I remember it, because it was the first time anybody ever called what I did sexy.

MOLLY WOOD: You're like, yes! [Laughter]

KEVIN NOVAK: But it was very, like, hysterical, right? And then, and the tide would go out. So I kind of felt like my entire operating career had been kind of living on a sine wave of hype. And so my belief, this is both how Rackhouse invests and sort of how I was thinking about this is like, in this environment, the sort of contrarian take in the classic, contrarian takes earn good investment returns, is just be steady. Check in with me in 10 years, I'll tell you how it's going. But my belief is over the next couple of decades that is good for returns, but I think ultimately healthy for the community.

MOLLY WOOD: What are you looking for? When you're thinking about industries with untapped potential to be transformed by AI, not to give away your whole secret sauce, but what have you identified?

KEVIN NOVAK: So it's interesting. One of the things I've always looked for when I'm talking to a founder-like, I really want to see like how quickly you can get to the core of what the idea is. And crucially, how quickly does it take you to get to the customer? Who is the person you're building for? And I think it's very easy to become in this sort of self-deceptive feedback loop of like, this is amazing because I love it, and then clearly everybody else will love it. But that leap is very rarely true. And so what I really love are founders who are customer-obsessed and industry-obsessed. And I can backstop that with a solid AI-powered, data-powered product, then Rackhouse will be very interested.

MOLLY WOOD: So then to flip that question, there's clearly the customer obsession, but one of the things we've been talking about a lot is this idea of AI-native companies and thinkers. Like, how do you describe the change in thinking that somebody who's thinking in an AI-native way might bring to their customers?

KEVIN NOVAK: One of the things I've often talked to my founders about is, do not overcomplicate it, right? And have a point of view on what the product should be, but do not be opinionated and bossy to your customers. When potential customers are presented with a product, which is doing an okay but not good job of meeting the customer where they are.

MOLLY WOOD: So to sort of put that in the language of the question, you don't have to come in and be like, I speak a whole new language. I'm an AI native. I have invented an entirely new thing that your risk-averse corporation is not going to want to adopt. You need to basically be able to say, I have a better tool that does a better job at the thing you need to do.

KEVIN NOVAK: Right. Especially right now, where you can say, look, if you get a compliance report for, you know, whatever your compliance task is, and you currently work with these consultants and they do 10 phone calls and send you a PDF, and those are the interfaces that you work with-an AI-native company, by the way, can build a product which does that whole compliance flow like, oh, this is intuitive and this is kind of a one-to-one replacement. The best AI products right now are following those AI-powered using traditional interfaces. I do think over time, like over the next three, four, five, 10 years, there's probably room for a completely, like, let's just abandon the idea of a phone call and a PDF, but I think that trying to hop, skip ahead in that way, at best it's inefficient and that you will spend more time trying to convince your organization to adopt than is probably rationally necessary, but I also think we as an industry, as a society, have not really yet figured out the working norms of AI. If you're living 10 years in the future, you might be making very wrong guesses about what the future ends up looking like.

MOLLY WOOD: Right. Well, I would also love to get your take on what role AI agents are going to play in this transformation of work that's unfolding already.

KEVIN NOVAK: Agents in general, for how I've been thinking about them, usually can handle tasks of such complexity that you need either a significantly longer chain of plans and steps, and/or they need to basically handle contingencies of what happens within this flow. So, for example, if I wanted to invent a new sport, which is some combination of football, soccer, and quidditch, come up with the entire rules for it, and then articulate the best possible robot that could possibly play this sport. There's a significant amount of contingent planning and all of these steps and saying like, all right, well, what's the shape of the ball? Well, I got to think about that whole thing. And what does that mean for a robot gripper arm or whatever, right? Like that level of-

MOLLY WOOD: Can the robot fly? I'm just, you know-

KEVIN NOVAK: Right, exactly. And if so, what's the process of locomotion, right? I'd say even now agents would probably struggle with that, but that's what I'm talking about in terms of contingent complexity.

MOLLY WOOD: Yeah. I feel like we've skipped a little bit over the idea of data integrity. How big a deal is that and/or potential barrier is that to rolling out some of these tools, or companies finding success with these tools?

KEVIN NOVAK: It's actually a very active area of innovation and research right now. I don't think there's consensus. What is fascinating is there's definitely an access and organization perspective, which is to say, you are usually coaching an AI on asking it to go out, take some of the company's written information, and weave that into a report or an essay that you're writing. And so making sure that the AI has access to the information you want it to have access to, and not having access to information that it shouldn't have access to-not every employee at every company has the right to every piece of information the company writes down. If you have factually incorrect information or problematic information in data sets, they will still use that. They have no natural internal barriers for disinformation or factually false information. So I think access, truthfulness, inclusion, exclusion-super, super important.

MOLLY WOOD: How do you think about the way AI is reshaping decision making? This is a little bit of an AI-native question, or even just sort of a thought process for you, for business leaders. When you start to incorporate the existence of this new tool, how do you think it starts to change the way you think about your business or solving your problems?

KEVIN NOVAK: I personally think that-especially when it comes to strategy decisions where there's not necessarily a knowable right answer beforehand-I still think that AI is at best streamlining those decisions. They are human-powered, they will be fundamentally human-powered. And I do think that that will probably be true, at least for the foreseeable future. One really good example of this, I was talking about this with a couple of VC friends, one of the things that we've been experimenting with here at Rackhouse has been, when we work with founders, one of the things that I personally tend to look for, not to give away too much of my secret sauce, but I look for leadership skills on the founding team. You know, every time I was meeting with a founding team, you scan their LinkedIn profile and you can make a pretty quick judgment call. Do they have the right management experience? Given that you can automate that-we're meeting with this founding team, we have their LinkedIns. You can feed it to an AI and just basically say, like, has managed equals true or false, right? That can all be automated. And then if you say, if everyone on the founding team is managed equals false, then what do you do about it? In fact, the way we run this is if that happens to be true in the first meeting, we bring it up. It's one of the first diligence questions in that first 30 minutes, because it's such a threshold issue for us. When I was talking about this with some other investor friends, I had other investors who said, oh, well, if has managed is false, then I wouldn't even take the meeting. And another group would say, well, if has managed is false, I don't care. I'd still love to meet with them. I'm actually really good at coaching people on how to be a manager. So there's this incredible amount of automation. You save time and you have the ability to have, I think, much more complex facts on hand when you're making strategy decisions. But there's still, I think, reasonable room to disagree about what is going to be the strategy that I don't think AI can reduce, at least not in the next 10, 20 years.

MOLLY WOOD: Yeah, totally. I like that, that it still at the end of the day will be, you'll get more information presented differently, but you still have to make a call. Well, so speaking of this mindset and the mindset of, again, the future customers for your startups, right? The adoption question. What is your advice to business leaders about how to develop a mindset, an organizational mindset, a personal mindset that makes them open for the potential of AI tools to come in and really change their business for the better?

KEVIN NOVAK: I would say that there definitely needs to be a culture of openness, experimentation, and engagement. I do think there needs to be an acceptance of, like, these trends are coming and they tend to compound, right? Organizations which begin to develop these muscles and are starting to hand off these processes. Not all experiments are going to be successful, but the ones that work tend to compound and save time and have a very financial benefit. The other thing I think that is interesting about where there's compounds, there's been, I think, a pretty reasonable dialogue around the sort of abundance narrative versus dystopian narrative around AI. Every time this comes up, I always counter with, well, think about your last five, 10 interactions with customer support. How many of them were objectively amazing? What I believe will happen are companies that are using this technology right are definitely going to automate some amount of the low-complexity work, and the best companies are going to take all that bandwidth back and raise the bar. I remember in the early 2010s, the one thing that you knew about Zappos was amazing customer support. That was the cornerstone of the brand. And I think we're going to see another wave of companies where 1) they will be kind of exceptional because of how they use AI with humans to deliver amazing customer experiences, but 2) I think it just raises the average level of competition. So even when you're talking about this as an executive, what I would expect is your competitors, especially the most AI-forward ones, are going to start raising the stakes of competition with you. They're going to market better. They're going to develop better products more quickly. They're going to deliver more customer support. I don't think it's overnight. They're definitely going to start making gains because of AI. How do we use AI to respond? How do we use AI to lead? In every organization, the humans involved are, by and large, hardworking, incredibly mission-oriented, incredibly intelligent, have long to-do lists of things they always wish that they could get to. Use AI to get to more of that.

MOLLY WOOD: You have talked about customer service specifically, and I've heard a lot of people talk about customer service specifically as an area with a lot of untapped potential. It's being transformed. It can be transformed but still has so much more potential for change.

KEVIN NOVAK: Yeah, so I think of every business function as sort of like "type 1" and "type 2" work. What I mean is, like, type 1 work is the thing that is attached to your job title. This is the thing that brings you joy. It's why you chose to become whatever you chose to become. You know, it's like, I want to be an architect because I want to draw amazing buildings and build amazing things. And then there's type 2 work, which is all the work you do about the type 1 work-you get the customers to pay you for the building you just designed. And so I think a lot of the type 2 stuff is very, very automatable with AI. Like, that's the kind of thing where I think an abundance narrative, like, AI takes over all the type 2 work in everybody's functions, and we all spend more time doing the type 1 stuff. Because a lot of it is incredibly fact-based and sort of information exchange-driven. You want to go from this city to that city, according to these parameters, and for some either failure of technology or some sort of weird corner case that the technology can't handle, you need to call a human being to have them overcome this burden. I think voice agents, AI, with that ability to respond and make decisions with these out-of-band constraints are very, very good at that. And so that the type of problem, especially the type 2 work that customer support teams wrestle with, is both from a complexity level is now achievable through agent-level AI, and then the medium tends to be voice, right? It's like, I'm usually calling in on phone or maybe I'm doing a chatbot. Both of those mediums are now intermediatable through AI systems. So I also think the sort of form factor is becoming AI-amenable.

MOLLY WOOD: I'd love to do a little lightning round with you. If you have a minute. First, I think it's important to say we have you to thank for surge pricing at Uber.

KEVIN NOVAK: That's right. Yeah. It's my contribution to modern society. You're welcome, everybody. AI in action.

MOLLY WOOD: Can you tell us about a specific way that you have recently used AI in your work? And then maybe an example from your life? I've heard you mention baseball a couple of times. I feel like that could come up in your answer.

KEVIN NOVAK: Well, I mean, as a good statistician, I'm a baseball fan. Like, every data scientist loves baseball.

MOLLY WOOD: I'm picking up what you're laying down. Yep.

KEVIN NOVAK: In personal life, we have two wonderful kiddos. Our oldest is three and a half and our youngest will be one next week. The three-and-a-half-year-old is a big colorer, big in coloring. And so one of my favorite things to do is we will go on adventures. We'll go to the aquarium or whatever Noah and dada do. And then I will go back home and then use an image generation AI and basically create coloring books of our adventure. You can literally describe the image of, you know, curly hair, three-and-a-half-year-old and a chunky bearded dad visiting the aquarium in cartoon, and then have it done in black and white and then he gets a coloring book of our adventure.

MOLLY WOOD: Um, I'm dying. My mom heart just grew three sizes. That's like the cutest thing I ever heard.

KEVIN NOVAK: Yeah. Totally custom coloring books through image generation.

MOLLY WOOD: Oh, I want to go back in time and make my kid little again-for lots of reasons, but especially that. Okay, so fast-forward three to five years. We've touched on this a little bit, but what do you think will be the most profound change in the way we work?

KEVIN NOVAK: Well, I mean, Rackhouse will be the most famous VC firm in Silicon Valley.

MOLLY WOOD: For sure. Obviously. Natch.

KEVIN NOVAK: Of course. Obviously. I think that the most profound change will be probably something like double-digit percentage of the internet traffic, like 10 to 20 percent of all traffic on the internet will be AI. You know, like, buy my kid shoes because he just stepped in a mud puddle or something, right? And I don't think the internet economy-advertising, SEO, whatever-is prepared for a world where 20 percent of the people on your site don't have limbic systems, for which ad impressions don't matter. Maybe they do matter. But I think it'll be the biggest thing, is the entire internet economy will be at least 15 percent AI.

MOLLY WOOD: Thank you again to Kevin Novak, founder of Rackhouse Ventures. Just sit with that, everybody. Just sit with that for a while.

KEVIN NOVAK: Thank you, Molly. Great to be here.

MOLLY WOOD: Please subscribe if you have not already, and check back for the rest of season 7, where we will continue to explore how AI is transforming every aspect of how we work. If you've got a question or a comment, please drop us an email at [email protected], and check out Microsoft's Work Trend Indexes and the WorkLab digital publication, where you'll find all our episodes along with thoughtful stories that explore how business leaders are thriving in today's new world of work. You can find all of it at microsoft.com/worklab. As for this podcast, please, if you don't mind, rate us, review us, and follow us wherever you listen. It helps us out a ton. The WorkLab podcast is a place for experts to share their insights and opinions. As students of the future of work, Microsoft values inputs from a diverse set of voices. That said, the opinions and findings of our guests are their own, and they may not necessarily reflect Microsoft's own research or positions. WorkLab is produced by Microsoft with Godfrey Dadich Partners and Reasonable Volume. I'm your host, Molly Wood. Sharon Kallander and Matthew Duncan produced this podcast. Jessica Voelker is the WorkLab editor.

Public link

Please use the above public link if you want to share this noodl on another website.