The Product Experience: a Mind the Product podcast

Building trust pays off - Simonetta Batteiger (Product Leadership Coach)

Mind the Product

Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.

0:00 | 36:33

Our Hosts
Lily Smith
enjoys working as a consultant product manager with early-stage and growing startups and as a mentor to other product managers. She’s currently Chief Product Officer at BBC Maestro, and has spent 13 years in the tech industry working with startups in the SaaS and mobile space. She’s worked on a diverse range of products – leading the product teams through discovery, prototyping, testing and delivery. Lily also founded ProductTank Bristol and runs ProductCamp in Bristol and Bath.

Randy Silver is a Leadership & Product Coach and Consultant. He gets teams unstuck, helping you to supercharge your results. Randy's held interim CPO and Leadership roles at scale-ups and SMEs, advised start-ups, and been Head of Product at HSBC and Sainsbury’s. He participated in Silicon Valley Product Group’s Coaching the Coaches forum, and speaks frequently at conferences and events. You can join one of communities he runs for CPOs (CPO Circles), Product Managers (Product In the {A}ether) and Product Coaches. He’s the author of What Do We Do Now? A Product Manager’s Guide to Strategy in the Time of COVID-19. A recovering music journalist and editor, Randy also launched Amazon’s music stores in the US & UK.

SPEAKER_02

Trust me for now until it breaks, right? Or let's disrupt it and then fix it later. That's been the motto of Silicon Valley for a long time and it got us where we are today without a deliberate approach to trustworthiness. You're shipping fast and you're shipping risk fast and it'll fall on your feet eventually. The CEO sacked 26% revenue loss in one week. If you extrapolate this to a year, that's a $580 million mistake as a result of some AI marketing up.

SPEAKER_00

If we can't define in our society what trust means, how do we codify that in our products?

SPEAKER_02

Actually, I don't think it's true that we have no definitions for these words. It's just not something that I hear product people talk about at lex. The principle I propose is trust over short-term profit.

SPEAKER_00

Simonota, welcome to the podcast. Thank you for joining us from Live backstage at MTPCon in London. Thanks for having me. We've known each other for I don't even know how many years at this point, but quite a few. And you are reasonably well known in this whole product community. But for anyone who doesn't know you already or didn't get the chance to see you on stage here at the conference today, can you just give us a little introduction? What do you do now and how did you get into this whole world of product anyway?

SPEAKER_02

Sure. I guess what I'm doing now is product leadership coaching. I'm also teaching workshops on commercial skills for product people. I think something that is becoming increasingly more important as uh business models shift. Building gets cheaper. We have to still make things that people love and want to give us money for and can trust and uh actually uh create delightful user experiences. And so I'm helping companies navigate this, try to make sense of what shifts and the way you need to think about building products as a result of AI and how to do it ethically, trustworthy, and responsibly. Um, something I've been caring about ever since I can think. I've been a product leader myself for 15 years. I built a marketplace for domain names, I built a platform for Cabos to run global payroll expense reporting, financials, compliance. I've built a feature set for ad filtering with IO that got built into huge browser projects, reached like 250 million users around the world. So I've done all kinds of product things, but before this, I was in sales, and before this, I was in finance. So a lot of the things that kind of shape the way I think about product now come from really truly understanding the commercial side of things and being in the room with executive teams when they make decisions about budget allocations, about investments into new initiatives, about how they think when they define what success means in an organization. And um, without ever thinking about this consciously, I applied this knowledge all the way through. And it ever, it always really helped me advance in my career. Because if you kind of like can't think like the CEO, speak about the numbers like they do, frame your proposals in strategic terms and and in commercial terms like they listen. So that's a little bit about um what I've been doing. I'm 47 years old. I like to dance tango, I like to eat well, I like to travel. Around the picture a little bit.

SPEAKER_00

But you said you you were in finance, you were in sales, and then you moved into the world of product. What led you down that path? What was your first role?

SPEAKER_02

That was actually like this crazy shift from finance to sales to product was all within one company. So I was hired there as the director of finance and worked directly with the CEO. And part of the mess I had to clean up in that finance team back then was like uh a mess in accounting that led to us moving from a very uh basic accounting solution to SAP, and that whole integration part of it showed the CEO that I can solve bigger problems and think in bigger things and think in systems. And then they had an issue with uh one of their sales team's leaders, and he's like, I think you can do this, I think you can figure out our approach for sales in North America in the same company and the same context. I already knew um what that company was doing, making and my colleagues. I had help from other sales leaders in the organization, and then they were like, Well, you know, we have this marketplace for domain names. We think people should also be able to buy these names through registrars around the world. We have this product for it, it's broken, we have no sales through it. Like, can't you figure out how to make this work? Fix it. Here's your new role in product. So I did that, and then I kind of rose the ranks in product in that org from individual contributor to team lead. Um, and from there on I kind of found my calling in product and stayed there. And uh nowadays, um, after exiting IO, like four years ago, I'm self-employed. I coach product leaders. I really like working with people around figuring out what is it that you actually care about, how can you put this in place? How can and this is akin to visioning, right? It's akin to how you think in product about strategy. So, like all of these things just came together beautifully. Here we are.

SPEAKER_00

So you mentioned trust earlier as something that you're really interested in. Uh I want to dig deeper on that with you today. So trust is let's let's start at the very basics with this, because trust, we all know what it is, and we know it's easy to earn, we know it's or excuse me, it's hard to earn, it's easy to squander, but I don't think most people would sit down and have sat down and really thought about what is the definition of trust, or what do we mean by it. So what do you mean by trust?

SPEAKER_02

I have two sources that I really like um in the way they framed what they consider to be trust. One is Stephen Kobe and his book, The Speed of Trust, and he kind of goes both in the direction of um trust has this dimension of um character, so your intentions and your integrity, but it also has this dimension of competency and results. So that to me was a little bit of an unlock because the way I used to think about trust before reading this book was oh, it's more like this wishy-washy thing. But if you think about it as a competency, like you trust a surgeon to perform the heart surgery, you shouldn't trust me to do this. It has to do with the competency of this person, translate this into an AI age. If you want to trust an agent, it has to have certain competencies. How does an agent earn competencies? You give it context, right? This context comes from somewhere, and this is how you start building trust in the products. So this is the way I like to think of it. And the other one that I really like in terms of definitions of trust is Charles Feldman. He wrote the thin book of trust. Um, another one that I really, really like, and one element from his definition of trust that I really like to bring into this thinking about how trust in AI and trustworthiness in AI plays a role for companies and teams, is he's very clear on the fact that trust requires an element of care from you towards the other person, from this other person towards you. So you would trust me if you believe that when I'm taking action, I'm also having your interests at heart. And I'm not kind of doing something that puts my interests before yours or doesn't take into account the impact of the things I'm having on you at all. And again, when I translate this into today's world and I look at the impact that tech products currently have out there, right? We have a whole new vocabulary of things around trust that we never even named three years ago. We have words like slop, hallucinations, psychophrency. We have uh words for um AI psychosis, we we talk about engitification, like you look you look at all these concepts, and underneath them is a violation of trust. Customers will not keep paying you if they don't trust you. And we've seen this so many times when things go wrong, customer loyalty shift. If they find any feasibly good alternative to your product, they will go there, right? We are seeing this right now on the geopolitical scale with some European companies all scrambling to try and find European alternatives to American vendors. And when you look at what just happened with Fable, right? Like decision by the US government to just turn a foundational model off for everyone who's not a US citizen living outside of the US. And I mean, eventually Anthropic will figure out how they're gonna do the user permissioning, so they'll give it back to the US users, but we will not have it here in Europe, right? At least not with current setup. This could be done with any system. Again, it's like who do you trust? Who do you build your dependencies on? Like, how do you think about this to make something trustworthy for your customers that reliably work, that will be working with their best interests at heart? And for me, this is the ambition I have for product, right? I want to be proud of the work I'm doing. I don't want to feel like I'm shipping something that takes my users for a ride. I'm doing something that I do the bare bones minimum to squeeze the most amount of money out of them and then hope for the best. People can look right through this. And I really think that um in the long run, if you center your decision making in trustworthiness, in explicability, in transparency, in keeping privacy at heart, in this care for the user, you're gonna be better off. That's where I'm coming from with this.

SPEAKER_00

Let's talk about trust as a metric uh in that we've you've in product we start off with discovery. We start off with the hypothesis, we want data, we want to prove things, we want to be able to show how something uh changes over time. Right. And so we need to be able to trust the systems that we're we're using for that. And we believe that there is a metric that we can do this based on. But then we look at things out in the real world, and we see, you know, from a few years ago it was NFTs, and you know, people uh f going in on top of that and meme stocks, and then you can say it with any number of different platforms uh and you know, people buying things, things go viral very, very quickly. Things rise up in value incredibly quickly or perceived value, and then the balloon pops. Right. So are we do we need to go for absolute trust, something that is stable, or are we is there a real market, or are we sometimes trying for that? Trust me for now until proven otherwise.

SPEAKER_02

You know, I mean, trust me for now until it breaks, right? Or um let's disrupt it and then fix it later. That's been the motto of Silicon Valley for a long time, and it got us where we are today. Some people got filthy rich on top of this, right? Um and good for them. That's just not my ambition for the world, and I don't think it's actually sustainable in the long run. My business is called inclusive leaders, right? I do care about inclusivity. I do care about treating people with equal value, and this trust topic is an extension of that. Do we need 100% perfection? Are we not allowed to make mistakes? No, absolutely we're allowed to make mistakes, right? But not negligently, not when we know better that something is trustworthy or non-trustworthy by design, right? AI foundation models are not trustworthy by design, right? And we have seen all the ways in which this can be broken. People using them as their personal coach, going into psychosis, killing themselves. Not an outcome we want to see in the world, right? In combination, robotics, automatic weapon systems, no human in the loop, and innocent people dying, and we're accepting some sort of a factor of a percentage rate of targeting mistakes in warfare. Not something I'd be proud of building, right? There are things where it doesn't matter as much, right? Even at IO, at the ad filtering business that we were working at, we had a machine learning algorithm that looked at this is an ad, it should get blocked, this is content, it should be let through. If this thing has a failure rate of 5%, you know, it's not the end of the world, right? It's good enough as a product experience still that our users were okay with us missing the mark occasionally and they see an ad that should be blocked. That's not so bad, right? But if the product that you're building now has um abilities to do things that you cannot control, that you cannot actually find good, you have to become more solid in your thinking about how do you explain getting from this input to that output. What kind of inputs do you not want to even see in your system? And you see this again, and Tropic is blocking certain requests for prompt input into their system because they don't deem this to be a trustworthy application of their system, right? So and we see on the output control side how sophisticated teams are putting e-balls in place and are very um deliberate in where they let an agent judge output or where they require a human to judge output. And I think we're gonna just get more and more um sophisticated and more and more solid in our way of actually treating this trust topic as infrastructure and not something that's like wishy-washy and hard to define. And you asked me about metrics, right? Um I do think if you have a good tracing of your outputs and you have good categorizations of all the errors that come out of your output traces, and you have an understanding of accuracy metrics, this is what the big players are putting in place, right? They know that they cannot launch a foundation model without this kind of thinking. And I feel like the hype that's been sold to many of us is oh, just use cloud code, build this thing, like this wipe coded apps, production ready, and then forget about compliance, forget about GDPR, forget about um transparency, forget about privacy, forget about all these things. It's just um not the way you can professionally build software and think that you can reliably make a living with it, because without a deliberate approach to trustworthiness, you're shipping fast and you're shipping risk fast. And it'll fall on your feet eventually. Look at what happened to Starbucks just two weeks ago with their tank day debacle. The CEO sacked 26% revenue loss in one week. If you extrapolate this to a year, that's a $580 a million dollar mistake as a result of some AI marketing fuck up.

SPEAKER_00

So trust comes in a couple of different ways. So if you're a big company and you've earned trust over time and you launch something new, there's an assumed or implied amount of maybe a trust umbrella that comes with it. It can be squandered really fast. You know, so right. Apple's maps were a joke for a long time. Siri was a joke for uh became a joke fairly quickly as LLMs came into play and showed how weak it was. Um, you know, any number I'm not just gonna pick on Apple, there are plenty of other companies. Amazon launched a phone that failed, and you know, any Google's famous for its product graveyard. Uh lots of things get launched and and don't work. But if you're starting out new on something, if you don't already have an earned table of trust to start from, um, and you're trying to stand out now, and you're trying to build something that's ethical and trustworthy, but there's lots of things that are launching on hype, how do you survive? How do you you know we we've been working together on this maker's manifesto for a while, and one of the interesting conversations we've had is it's not just enough to build and launch the product, it's about usage, it's about distribution, it's about uh adoption. How do you get to that point through a sea of hype?

SPEAKER_02

You know, honestly, I think this hasn't drastically changed because if you are actually solving a real user progress desire in a good way, people will tell each other about it. So if you actually have product market fit in the form of a desperate customer who you are so delighted with your solution, right, they will tell five other people about it. That's how you keep standing out, and that's the same way it has always been. Now, obviously, if you have the existing distribution channels of a huge platform like Google or Apple or Microsoft or Amazon or Netflix or Spotify or you name the big name B2C players who have huge reach into their customer base already, they have an advantage, right? But then again, you know, three years ago, no one knew lovable, no one used cursor, no one used even Anthropic as a company is actually fairly young, right? So they were in the right place with something good enough that people really desperately wanted at the right time and grew out of lots of people excitedly talking about it and solving a real problem, right? I mean, what coding agents are solving, and I think um Claude Code in particular and Codex and a few others are really good at this, is this really expensive thing of building product, right? So any coding agent can now write the same amount of coding solution in days that used to take weeks and months of teams to build. Obviously, companies are excited about that, right? Obviously, they want to have this tool on hand and make use of it. I've played with it. It's really fun, but I still think that whatever comes out of my dabbling with it in a hackathon weekend without all of the really good guardrailing systems around it that a professional software engineering team would put in place, this thing isn't production ready. It's not trustworthy, it doesn't scale.

SPEAKER_00

Let's talk a little bit more about that. Because, you know, if we can't define in our society what trust means or what fairness means, and we're seeing a huge divide in that society always, how do we codify that in our products?

SPEAKER_02

Actually, I don't think it's true that we have no definitions for these words. It's just not something that I hear product people talk about at length. But just like, I mean, literally go to Brene Bram's podcast from the last five years, pick out the episodes that speak to the experts that have written papers and books on trust, on fairness, on inclusivity, on transparency, and on all these topics. These are not like unsolved problems, undefined things. It's that we often just kind of like got lucky that in the way we used to build things, we had human decision making at so many points in time where we kind of took advantage of the fact that most humans are moral human beings and think of what their impact of their product would be and what a good user experience would be. That's what these designers and engineers and product people implicitly put in place in all of the deterministic software that we've been building, right? It's like concepts we apply almost without thinking about it because we learn them growing up in kindergarten, right? But it is not true that we have no formal definitions for these things, and you just have to go and find them.

SPEAKER_00

I don't know if it's the definitions, I think it's the interpretations potentially. Because you know, in the US right now, it's it's 50-50 on some of these things about whether the politics that I suspect you and I share on some of these issues are the right way to approach it versus uh what I'm seeing from, say, you know, on Mosca and Palantir, and that's my personal opinion on these things, uh their approach to these things. Here in the UK, there's a significant amount of the population that votes one way and a significant amount that votes a different way with regard to what they believe is ethical with regards to immigration. Same thing in Germany and other places. So I yes, we can have a definition, but the interpretation is something that is very, very debated and subject to a lot of things, a lot of uh um consternation. I I'm not sure how we codify it.

SPEAKER_02

And that's fine, right? And and to an extent, that has also always been the case, right? We have always had um Opposing viewpoints on almost anything, right? I still think you can have a point of view, and you can lean on the research that shows how certain things actually lead to very good outcomes, right? Most of us have seen the research Amy Edmondson has done on what makes a high-performing team and how she named psychological safety as one of the main driving factors into good outcomes that teams create together, right? Likewise, we have tons and tons of research that amping up diversity is one of the main things that leads to better innovation and to better decision making, which leads to the fun fact that in more than 90 shareholder meetings in the United States, where right-wing people tried to put motions forward that you should get rid of diversity rulings as part of board considerations, they were voted down because shareholders and boards actually understood that that would be detrimental to their business. So I get it that the public debate seems to be super polarized. I also know what leads to successful outcomes. And the research on that's pretty clear. I've yet to see a piece of research that says more polarization, less diversity in decision making, less trust in teams would lead to better outcomes. I just don't buy it, right? So, I mean, sure, if you want to have this point of view, you want to experiment with that, you want to try it that way, go ahead. That's just not the people I work with. This is not how I want to build products. And when I care about a world in which joy is present, connection matters, caring for other people matters, trust matters. I want to buy things that I know will work. I want to buy things that I know will not harm other people. I want to give my money to companies who I believe care about human beings. I want to give my money to companies that don't believe that uh the sustainability of our planet is uh a stupid European idea that's not worthwhile entertaining. I'm gonna vote with my money too. And I see lots of people voting the same way. And you know, let uh let people's budgets decide.

SPEAKER_00

Okay, let's let's move on to a slightly different topic related to this. One of the key things you were talking about today was designing to prevent the atrophy of skills. How does that play into all this?

SPEAKER_02

It's just something that I've, you know, I'm watching the world, right? I'm watching what are people actually worried about when it comes to AI. And this is one of the worries, right? Like, how do we keep people that enter the workforce today able to learn the skills that we have learned? Everyone names judgment as and taste as the things that we now bring to the table. How do you develop that? You don't develop that if you don't develop your skills, right? If your skills get worse at something, you're actually not getting to better judgment, not better taste as a doctor to worse diagnostic skills, as a pilot to worse ability to judge what a situation is dangerous. And I think this matters that we don't lose our skills. And so I think we can build workflows that preserve skills, that put the right amount of friction in to have someone actually enjoy doing their work and not just become the quality assurance person of some AI process, which nobody enjoys. Designers don't enjoy this, engineers don't enjoy this, product people don't enjoy it. The fun is in actually creating something and seeing the efforts of your work have a positive impact, right? Um, so that's part of uh where this is coming from. And I really think there are nice ways to have AI help us become better at thinking, help have AI question us in the right places where thinking deserves to be challenged. And as a result, then we learn and we grow as people. And again, this is just something I think uh we can do, should be doing, and that's what I talked about today as one of the skills that we can bring to the table.

SPEAKER_00

So let's talk about developing this skill a little bit. Um, from uh you have a background in finance, you work with people to help communicate the value of the work they're doing in terms of business metrics. Most of the time, our dashboards, the things we're focusing on in measurement, they're very often output focused. Uh this outcome is a lagging metric, it's not something that you can control as much. And we do that all the time. We want speed, we want efficiency, we want number of things, widgets processed. Everything else comes downstream from that. This it's to a certain degree, it almost feels like the accessibility arguments. You know, this is important, we know trust is important, we know skills development is important, but in the short term, we were trying to maximize for something else because that's what shows up on the dashboard, that's what we can measure. So, how do we make this argument at the boardroom level to say, yes, we want to maximize the outputs, but also really focus on the outcomes?

SPEAKER_02

I mean, honestly, boards ultimately care about the financial outcome and success of the organization that they're overseeing, right? They also try to keep them compliant. So I don't think that boards are too hard to convince that they wouldn't want to be governing something that leads to legal risks, shitstorms in social media, and a company's revenues collapsing as a result of poor trust in the company. I mean, again, to the Starbucks South Korea example, right? Like they care about the 25% revenue loss, right? And yes, you can say that's like a super lagging outcome-related indicator, but that's ultimately what we're responsible for, right? Like as product people, if you boil it down to the basic basics, it's like create value for your users, create value for your organizations. Those are the two outcomes you're being judged by. If you deliver that, that's like a successful product you've built, right? And I think for a while you could rely on people's um moral compass in human decision making well enough that you didn't have to have an intentional approach to trustworthiness and ethics in your products. But if you have agentic co-creators in the game, they need explicit roles and definitions of what you consider to be good vision success, what you consider to be good strategy success, what you consider to be good financial outcomes you want to optimize towards, and also what trustworthiness really means. Because if you don't give them that rule set, they will just do something random.

SPEAKER_00

From the big picture, I totally understand that. At the smaller picture level, at the day-to-day decisions, you know, lots of companies are dealing with huge financial pressure. I can remember a company I was working with, um, a large retailer, had a challenge. We were going through the budget for everything. We were doing zero-based budgeting and going through every cost item. And one of the things that uh came up for review was a uh uh pornography filter for the uh items on display at the stores. And I was trying to argue is this doesn't cost anything in the grand scheme of things. It's we're spending more time arguing about this than the in terms of value paid to us than this license cost. And the reputational hit on this, if something went wrong in one of our stores, is massive. So why are we arguing about this one? Why do I really have to justify this one? So every single decision on a day-to-day level seems, you know, comes under massive pressure. And it's what is your risk acceptance around these things? How do you how do you it's big picture board level, sure, but how do you operationalize that every day and say, right, this is the level of risk we need to imbue trust and accept certain costs around certain things?

SPEAKER_02

I mean, to me, that is a leadership task, right? So if you are the CPO in your organization, if you're the most senior product person you're in your organization, you choose to embrace that and you bring this into your product org, right? You sit down with people, you actually speak to them about this, you make this a priority. If you don't, you go the by default, non-deliberate, risky approach, which is a choice you can make, right? Or maybe you didn't even intentionally choose it, you just happen to float down that path, right? But that is to me like leadership, right? You define what you consider to be a good outcome, a good impact towards, which you want to optimize with your team, right? And honestly, that is like product leadership 101 to me again, right? Because you are envisioning a progress for your customers in the world. And part of this progress for your customers in the world is like, let's not make it risky for them. Let's keep it safe, right? Let's make it delightful, right? What Nasrina's speaking about in her book about delight, I honestly think that's what can set you apart, right?

SPEAKER_00

It sounds like uh to borrow from from Martin Erickson's book, this needs to be part of the fundamental principles that the strategy sits on top of. So the rest of your decision stack has to have this kind of stuff baked in for it to be successful.

SPEAKER_02

Actually, I I had Martin Erickson's um decision stack in my talk, and the principle I propose is trust over short-term profit. So it's visible there, and people can stumble upon it and be like, what does that mean? So you are in the conversation. That's a very easy way how you can start operationalizing it. And then I showed another slide from Prompt Ledger, really goes into like many layers of this architecture for trust that you can think about and employ and put into action. All the way, starting from like how do you ID an agent that takes any kind of action, how do you bake in explicability into your um product? And I think for the first time I am seeing a cool use case for blockchain there, where you can trace which input related to what decision making, what output, and keep a score of it somehow. Um you have to think about like what do you do if a user says this isn't cool? What do you fall back on? Like there's so many ways in which you can operationalize that, right? I published uh a canvas a year and a half ago on how to operationalize ethical thinking into products, not just AI products, but also AI products, right? There's tons of resources out there, and you just choose to want to go for them.

SPEAKER_00

So, do you think that all the AI stuff where we're going to should have an ethics.md file baked into it?

SPEAKER_02

I think the companies you want to trust eventually go that route.

SPEAKER_00

Have you seen any examples of it being done well?

SPEAKER_02

I mean, I don't have full insight into the whole story, right? But I really like how Eric Reese in his new book, Incorruptible, speaks about the governance structure of how you start from the very beginning before you even collect your first round of funding, um, how you preserve the ability to make core decisions, make ethical decisions, put an ethics board in place. And he quotes Anthropic as one of these examples and did this really well in having encodified in their in their governance structure that board seats in ports have to be filled with the from this ethics committee, from people that cannot have full chairs in the company at all. So that even on the governance layer, there are things you can do, right? This is not the thing that the average product person who gets employed in this organization gets to choose to do. This is what founders might put in place, right? But you can ask questions about the company you're about to challenge. Do you do you have that? Why don't you have that? It might kick it into gear that people will put stuff like this into place. If you found yourself and you started up your new thing from scratch right now, you can put this in place from day one.

SPEAKER_00

And hopefully that's a little more sustainable than just a motto of be saying don't be evil.

SPEAKER_02

Which got strapped. Yeah. Yeah.

SPEAKER_00

So Simonetta, this has been fantastic. Thank you so much for doing this. One last question on this. For someone who's trying to get started tomorrow, and you're not at the leadership level, I think you made it really clear what to do at a leadership level. But if you're a mid-tier product person, what's the one thing that the I can do tomorrow?

SPEAKER_02

You can start a conversation and ask curious questions and bring an article like the Starbucks thing, bring Eric Reese's book, bring my canvas on how to operationalize ethics in your AI products, and just say, let's talk about this. How might we do this here? You have a voice. You have agency. You can make ethical decisions without anyone's permission. Use your agency, use your voice, be curious, start conversations. Anyone can start.

SPEAKER_00

Fantastic. Cementer, thank you so much.

SPEAKER_02

You're welcome. Has been a joy.

SPEAKER_00

The product experience hosts are me, Lily Smith, host by night and chief product officer by day. And me, Randy Silver, also host by night. And I spend my days working with product and leadership teams, helping their teams to do amazing work.

SPEAKER_02

Lou Ron Pratt is our producer, and Luke Smith is our editor.