Taking the Busy Work Out of Building Data Pipelines at Scale w/ Chris White

ABOUT THIS EPISODE

Within a year of the first prototype, Prefect had their first paying customer — a Fortune 100 company. Their SaaS offering was not even publicly available yet.

That was partly because they were developing a platform that allowed companies to build, run, and monitor data pipelines at scale, a problem no one else had tackled in quite the same way.

Another reason was their commitment to the iteration cycle during development. By releasing the MVP early, they were able to gather feedback immediately and improve the product with incredible speed.

In this episode, Prefect’s CTO Chris White shares the challenges they encountered and lessons they learned along the way.

We discuss:

- How Prefect helps developers run and monitor data pipelines at scale

- Their strategy around open source

- Intentional choices made during their development journey

- Building a culture that embraces mistakes

Check out these resources we mentioned during the podcast:

- Find design books by Sandi Metz at https://sandimetz.com/products 

- Find Creativity, Inc. by Ed Catmull at https://www.creativityincbook.com/about/ 

Want to hear more stories from high growth software companies? Subscribe to Application Modernization on Apple Podcasts, Spotify, or check out our website.

Listening on a desktop & can’t see the links? Just search for Application Modernization in your favorite podcast player.

You are listening to applicationmodernization, a show that spotlights the forward thinking, leaders of HighGros software companies from scaling applications and accelerating time tomarket, to avoiding expensive license and costs. we discuss how you caninnovate with new technology and forward thinking processes and savesome cash in the process. Let's get into it. Thanks for listening to shatesapplication, modernization podcast, I'm your host Nick Mark Arely, the goldestpodcast, is to focus on speed scaled savings related to the stories of highgrowth software companies. Today, I'm speaking to Chris White, the CT prefectthey help customers build, run and monitor data pipe lines to scale. Ourconversation will cover how prefect came about building teams and technicalinflection points in the development of their offering thanks to red hat forits continued support of the application. Modernization podcast, HeyChris thanks for joining us today, yeah thanks for having me neck happy to haveyou. So we came across your your organization and I was like a this. Isthis? Is the type of people I'd love to talk to? And you know we had a quickconversation and sounds like you're doing some really innovative work overyour organization. So why don't you give us a minute explanation of whatyou guys do and what you're trying to solve for customers and Witler Ismen onwhat you do a prefect sure, so the the really short version is prefect allowsdevelopers, especially in the data space, to build, run and monitor datapipe lines. It scale, that's the like the headline, but kind of getting alittle bit more into the weeds. What that really means is we provide thingslike highly configurable scheduling of Pipelines, observability of YourBusiness Logic, so you know the different tasks that might make up aworkflow and then a lot of other features that we usually frame asnegative engineering, so kind of a way to think about that is, it'sessentially all of the Defensive Code that people end up writing and thedefensive infrastructure that people build in order to anticipate unknownfailure modes. So, for example, you might put in a bunch of logging becauseyou're not sure you know where in a task something might go wrong. Prefectessentially handles that all for you kind of out of the box, so you canfocus on writing what we would consider. You know the innovative code that youwere really hired to do sprinkle in a couple of prefect idiomsthroughout, and you know, you're essentially good- to go to productionize your data pipe lines. Gosh is so you kind of take all the hard work outof building and maintaining thatinfrastructure specific to running dead, to pipe linesand being able to extract useful information from all that data. Yes,and there is a little bit of a catch, that's probably worth calling out,which is that we don't actually manage and fostrow you and that's actually afeature, and you know we can get a little bit into that end. We talk aboutsome of the challenges we've overcome, but prefect. You still own yourexecution environments, where your pipe lines are running. So that's totally onyou. You can lock them down. You know, however, you need to for youroperations and they communicate with our API to essentially send and receiveinstructions bits of Metadata that you might find informative and our dashboards that you can know queery and filter things like that, and so so yeah,it's a big part of it. Is this observability piece and then kind ofunderneath there are a lot of these other things like scheduling, cashing,for example, so efficient you know, detection when it past doesn't actuallyneed to rerun things like that, got it and yeah and the way the way it's built.This is probably also worth falling out. Is We have an open source package,that's written in Python, and so that's how you you know, integrate prefect isvery much a code for solution, so you...

...can write completely arbitrary pythonlogic and then put prefect around it. Our focus is very much on the dataspace but in theory there's nothing that would prevent you from building.You know essentially anything with brief act. I mean we run slack potsinternally using prefect, just because it's fun, that's interesting. So arethere any kind of commercial use cases that your customers are using outside ofjust data pipe lines yeah actually at least so the most interesting one?That's top of my mind. Lately I've just been interacting with them. A lot isorchestrating the building of various python packages across diverseoperating systems and different machine types, and so that's a type ofscheduling. That's not necessarily time based it's location, based quoteunquote, where you know you have one work flow you want to parameterized bymachine type and then actually have it run on the different machine types tobuild these package, artifacts and- and this team uses prefect e to managethose pipe lines and, to you know, monitor them and etc, and kick them offinteresting. So would you say that the way you you guys have designed yoursoftware is allowing customers to do that, or isthat something you ever like? Oh, we could probably do that. Let's go builda feature on this or is it kind of your approach? Yeah,that's a really good question. We try really hard to strike a balance, and Imean I'm sure everyone does this to some extent right between what we knowour immediate user base is, and that's the data community, but making surethat the concepts are and abstractions that we build are general enough sothat our users, you know, can can kind of reconceive their world and prefectlandscape and build more interesting things. So I think for us somethingthat we talk. A lot about internally is just like getting the right abstractionwhenever we're building a new feature, so that we're not overly tending to oneuse case, but we're still, of course, satisfying some concrete use case, butwith the ability that it might get extended by the community andinteresting ways that we can't predict Goshi, that's pretty cool yeah, it'svery fine! So when, when did you join prefect, were youground floor? Did you join recently? I was ground floor, so I've been withprefect since day one it's about three and a half is years now I was the firsthirer with a CEO and founder Jeremiah and essentially you know, built built aplatform built to open source package with with him and now, of course,oversea development et CE GOSH is so so was there a strategy around open sore,or is that, like just your preference, where you came out of and you're likewe're going to do this up in source? Like? Can you talk us through some ofthat? There's a lot. A lot of data data tools did a platforms. A lot of themare proprietary, they maybe even start with open source, and then they getlocked down and they become this Fremi or you know an open core kind ofwalk me through some of that journey as much as you can yeah yeah. This is isan interesting topic. So one thing that's a common misconception aboutprefect is that we were first in open source package that we then decided tobuild a company around, and that is not true. We had a business plan now, don'tget me wrong, we've changed it multiple times. You know, as we learn new things,but we had a business plan before prefect was ever open sourced or apatchy to license. So we thought a lot about you know. Why are we doing this?What are the benefits? What the risks things like that and a big part of itis pure ethos. We believe in open source software. We believe in thepower of open source software to create you know powerful communities that helpeach other. So that was a big part of it. Is We wanted to create one of thesecommunities and do so? You know in the healthiest way that we could so I meananyone in our community. I think will tell you that it's a very upbeat,positive, helpful community, and that is really been a focus of ours- ismaking sure we can maintain that tone...

...so that everyone feels welcome and theycan get their questions answered efficiently and you know learn aboutour product efficiently and we really try hard to ask ourselves the there's,a difference between the code that we write and the value that we deliver andso the more code that we can put out intothe world as open source code for people to build on, extend and workwith the better. You know the value that we deliver, especially to thingslike enterprise companies are largely around security and off, and thingslike that, and that's not something. You know that stuff that we keep aprietor for a lot of different reasons, but one of which is so that we can takea very strong, firm opinion on what it looks like and you know, keep it to thequality standards that that we wanted to be at another thing too, on the opensource front, I think, is important to note. Is We have a really strongopinion and I don't think this is controversial at all that any developertool you need to be able to debug it when itgoes wrong and open source is an incredibly powerful way to aid in that,so you can look at a trace back. You can go into the codebase. You can findthe line of code that aired out. You can drop your own print statements init because you can fork it and you can, you know essentially get yourself sofar and then prefect cloud, which is the the SASS offering that we have isessentially an API and so as long as we document those that interface and therules for that API. Well, we have a really good covering so that users canself debug and I think that's incredibly important, especially givenour value prop that you know we're trying to take care of that defensivecoding. For you right. That's that's interesting. You know, you know we livein open source every day here, shot us off. So that's where we start our business, where wecontinue to make investments with customers. So it's interesting to hear why, as a product company, becausewe're not a product company, we're an integrator to hear why you why youwould make those types of decisions, because at the end of the today we haveto advise customers on what their strategy would be for whatever it is,whether it's you know, analyzing data or shipping things worldwide, and you know when we can make decisionsthat help them take advantage of an offering and and think more about what they deliverto customers and it's transparent and it's open. They win right, everybodyreally wins so yeah. I agree tell me about what was the bug to star prefect? Yes like: Where did youguys arrive and you go? You know, there's a thing out there that we needto do for for people, maybe for us and then eventually for people. How did youguys arrive there? What's the story there? So there are a few different mile stemshere. I think the the biggest one and the most central one it really startswith Jeremiad a founder. He was on the he was on the original P MC for apatchy air flow which is or was a really popular, open, work float toolthat came out of Arab and B many years ago, and so he was a part of you knowgetting it into the pathic combat on etc, and he, as a data scientist, wasreally trying to push the limits of air flow, and understandably, he kind ofhad to give up on that. He wanted, for example, in memory data passage,parameterization running things at scales, sopotentially hundreds of thousands of tasks and in a given a workflow andaraphoes architecture. It was designed for her do, but just does not supportthat in any way- and you know he definitely poked out it and tried, butit it kind of fell, fell on its face, and so he started building thisprototype purely for himself inside his company that handled some of thesethings. So it definitely wasn't production ready, but it was just kindof a little proven concept and someone in his net network essentially said Hey.That looks really interesting. I would invest in that and so kind of veryorganically. He started considering...

...that looking around deciding you knowif there is actually a problem to be solved here and if there is aresounding yes, airflow is very popular, as I'm sure a lot of people listeningknow, but there are very few people in the world that enjoy air flow. They useit because they have to not because they want to so there is definitely aproblem to be solved. A ton of features that air flow just could not support inthe data workforce, and so you know with that prototype he started choppingaround and then he found me because I had written some dogs on desk and wereally wanted a native desk integration here, and so we started started talking,ripping ideas off each other and very quickly. You know landed on this kindof negative engineering concept and then went full bar with that, soscratching your own itch as most things ye right. It's like thisisn't go at what exact me see if I can do something a little different and Oh,hey, here's the thing: here's a company, here's a there's, an idea that mighthelp people- and you know- maybe there's a business around this. Sothat's really great awesome. So why don't? Why? Don't you tell us a littlebit about? Maybe the journey around how you guysare developing your sophomore, maybe like a short history of we started here andprobably for the technical in mind. Sometimes we're a little abstractedaway from like the deep technology and I'm not an engineer so be gentle butsure sure, but you know maybe the high level concepts. If you know we starteddoing things this way and maybe the pivot point you had along the way wereyou like we had to make a different decision yeah. So our journey has beenvery interesting and I think the story I just told you about how prefacingcame into existence is a theme which is that we have kind of succeeded, despiteourselves in a certain sense, and that is not meant to pat ourselves on theback I'll explain what I mean bit that I just think that we are titely solvinga problem that no one else has focused on in the way that we're focusing on itand so, for example, our first paying customer was a fortune one hundredcompany, and it was before we ever had an actual public saft offering, andthere were just kept saying we want to pay for this. We want to pay for thisand we literally had to tell them there are certain buttons. You cannot click,because you might break something and they were totally fine with it, and sothey became our first paying customer and it was before we ever even had hadany plans. For that I mean we were still very much in the early days, butthe observability scheduling and just the user experience of building andscaling work case was enough for them to to want to engage with us in aformal way. And so a lot of our history has kind of been like that, and so oneof the pivot points. They were actually inder going now, and they can tell youa little bit too about some of the other problems that we've run into. Butwe have three and a half years of essentially hard won knowledge from ourcommunity of what features, and you know how the Argono ics of how peopleexpect these things to work, and we are kind of revisiting a lot of our coreproduct assumptions and I don't want to say re architecting entirely that mightsound scarer than it needs to, but but essentially rear Chotei. How prefectwork clothes are built and executed to satisfy this three years of use. Casesthat we've seen that are kind of on the edges of what we're capable of rightnow so that that has been yeah. A big focus of ours lately and a part of thisis kind of how we like to develop, especially in the early days where gaining knowledge is really the mostimportant thing for us, and so we really like to release you know, get alot of feedback from our users figure out. What is the common pattern here ofwhat our users are saying? They need or want, and then figuring out someminimal version that we know we can support. You know it's definitely areal feature, but also still minimum viable, not necessarily like fully. Youknow, fleshed out, get it in the wild and see. Does that resonate? Does thatactually solve the problem and kind of...

...continuing to do that and no surprisewhen you do that for a long time, you do end up with a little bit of apatchwork that might not always come together in the right way, and sothat's kind of where we're trying to revisit and come up with a much moreprincipal approach to what our software looks and feels like, and you candefinitely expect to hear some noise about that in just a few months. Moreinterestingly, I guess. On the technical side we had an early advisor,and this was when your first specking out the business model before prefectwas ever open source. Who Essentially said. I really like this problem thatyour something it makes a lot of sense to me and I could definitely use it.However, I already have a text tack and I really don't want to bring in someother new technology into my text, sack that requires me to like re architect,my own things. So that's the first challenge. The second challenge is, butI also am not going to give you either my code or my data on any sort ofmanage service, because I just have extreme security requirements, I'm inten text, space etc, and he said so. That's the problem that I give to youand we thought for a long time like yet. Does this make sense? Is this acomplete nonstarter? Is this even possible- and this is when we came upwith what we call the hybrid model for prefect- It's not particularlyinformative name, but the idea is it's: This open source python package, that'sthe client! You build your workflow using that you can test it run itlocally, but anything involving state or state fulness goes through an APIand kind of one of the key insights is that all orchestration, retry, cashingall of that stuff can operate on very mild Meta data. So we might need thingslike file names. So there is, you know a little bit that we will get, but wedon't actually need the data that stored in that file to know that heythis task is rerunning. It actually can pull a cash value and that cash valueis located in this location. We prefect do don't have access to it, but yourpliant code does so it can get that location from the API, pull it locallyinto memory, etc, and that was actually a really hard problem to solve.Initially was: How can we do this, so we don't even see your workflow codeand anyone us some or the prefecto that we have all these concepts of flowstorage and things like that and a prefect agent, and all of that is inservice of this separation between execution and orchestrationobservability, and there are a lot of really hard problems. So, just as anexample, how do you detect process death whenever you don't have a routeinto someone's execution environment and the answer not particularly surprising, but takesa while to figure out how to do efficiently, which is you sendheartbeats? And if the heart beat stops, then you know something's gone wrongand you need to tell the agent to try to resume the work blow or checking onit or something like that, and there are a lot of other kind of chicken andegg problems like that with your flow code. So how does the agent know whereto find your workflow code, because we don't have it you put it somewhere andmaking sure that our versioning behind the API matches the versioning you'redoing client side just a lot of things like that that were that's o Beinhonestly, don't think I be t yeah, yeah yeah it's and I can see how that canresonate with certain types, commercial customers. You know Fintech. Obviouslyyou know everybody's worried about a big one. Yeah everybody's worried aboutlike how much how much access do you have to my data, which is my livelihoodas an organization. It's private, it's Pii, you know were worse so right. That'sbent that that seems like a big challenge initially to try to solvearound so picky backing off that when you gotstarted at that customer that you mentioned. That said that we're cool well e. We want touse your stuff. How far along in the...

...journey was. That was that less than ayear it sounds pretty rapid yeah. It was less than one year and so big bigmiles sin for us with that first check, and then we worked, we still kept it indata. We worked more closely with them. Just to you know, make sure that we didallow for every button to be bressed pretty quickly. A hot became our focusfor shore, and this is kind of actually a theme of how we really suffer ingeneral, and you are going to see some Hensius later this year, where wereally like to have a close set of partners prior to makinganything public just to get some of that really and depth feedback andmaking sure that you know we are in fact, like. I said S, solving a realproblem for people delivering value and then, after we kind of go through thatrefining, have a larger public release for some of the lessons, learned, etcand then continue to iterate in the open with the community, and you knowall the diverse set of users that we have have in the community right now.That makes sense, because I think you know, as you iterate, through buildinga product, an idea doesn't mean it's a product. It just means it's an ideauntil you can prove it out with some some opinions from people that arewilling to pay for it or you know, invested in with time. So you know wesee open source projects. We see you know massive community involvementaround things that are impactful to that community and usually that ends upturning into an enterprise offering at some point so yeah. No, that's! That's! That'sInterestin! THAT'S QUITE FAST! That's pretty impressive, because you know yourun into a customer who wants to give you money. You know a year or less in I meanthat's, that's quite a quite a hurdle to get through and it's reallyimportant to and this. This is true both when you're looking at yourcommercial offering and in your open source, often that you don't let thatone customer drive your product in a vacuum right, you're, not building itjust for them, although of course you are building it for them, but with alot of other people in mind, and so it's really important, especially inthe early days, but I mean that I think this is universally true to just keepyour eye on what problem you were trying to solvekind of, even independently of that first customer and making sure that youknow you have a good quality gate for whento make decisions. So, just because a customer request, a feature doesn'tmean it's a good idea to built the feature, for example right absolutelyso as an organization, you know you leading the you know the development practicearound the product. What are some non technical challenges you've run in over.You know three and a half years. Usually this revolves around building ateam and things like that. But you have any lessons learned around that side ofthe business I do. Actually yeah hiring is hard. That's. I do need to talkabout that. I think everyone knows that Yep. I think for us. It's a lot aboutculture and making sure that we really thoughtfully considered our culture.Even when we were a very tiny team so that we could scale it because there'sthis mysterious point that happens where all of a sudden you're biggerthan you thought and maybe don't have a close relationship with every singleperson in the company any more, and so it becomes harded and retroactivelyattempt to create a culture. I mean a set of values and standards, and so wereally we would iterate on on culture. You know, values and standards, how wework documents like that, even when it didn't make sense, because there areonly maybe five people working on it. But that way whenever we did hit thatkind of inflection point and started to bring a lot more people in. We had avery thoughtful answer to just pretty much everything, and so we had verystructured on boarding process from day one where we send you through a set oftrainings. And you know you had ninety bay plans and like we did all thisstuff that I don't think is typical at a company of our size when we weredoing it. But it was an massive massive value ad for us, and I think anotherpart of this is making sure, especially...

...at a start up that you have a culturewere making mistakes and learning is valued. Actually, when you make amistake that that's an opportunity to learn- and I don't think it makes senseto have an empowerment without the ability to make mistakes and failsometimes- and so you know- you'll have developers, maybe on the more juniorside that are just petrified to do something or really something to pride,because what if they mess something up then- and we just have to remind them-that hey we're all going to make mistakes here- the catches that wedon't want to make the same mistake three times in a row. We always want tobe making new mistakes, and that's actually really hard to to get peopleto everyone will mid. But to really do it in practice can be a little bit morescary. I think for some folks. Well, you have to be very intentional right,so in order to enable your development team to live on theedge and maybe screw some stuff up, they've got to be able to do that inthe way e that it's not completely ruin the product right, so it gets down tolike the you know, you talk about on boarding. You know we have a way to doon boarding, so everybody knows what's expected and where they should do thatand how they should act and who they should reach out to when they needsomething. I see that is one of the biggerchallenges with customers with their development teams. They're like I can'tset this thing on fire, even though I probably need to put this in a testenvironment and set it on fire just to see what will happen. So you know theso they're just like making small incremental improvements instead ofgoing. Maybe this is trash and I need to rethink the way. This looks yeah andI to be like a little more concrete. One way that you can achieve this isthe leadership team making sure that they are willing to admit when they'rewrong. You know internally, and so, as an example, we've changed our pricingmodel. You know a couple of times over the years and well course have apresentation. Justification for the pricing model explain the reasoning whywe definitely think it's the right fit like very confident, etc. And then welearned that we were wrong and that's totally fine, and so then we now make sure that everyone's on board-and we give the the next follower presentation a year later whenever itmay be explain what we learned what mistakes were made and why we'rechanging the next one and just never being afraid of just revisiting everysingle decision and openly doing so and then the more you do that the morepeople start to realize like hey. We are all experimenting together andlearning from each other and that's totally okay. As long as we do it, youknow intentionally and thoughtfully along the way right yeah. So if you're,if you're, showing by example a culture, that's willing to learn evenwith something that's you know, maybe from an engineering perspective, thenine is pricing right. That's a sales thing sure yep, but having engineeringunderstand why you're making those decisions, because they don't theyprobably don't care if it's five dollars a a user or five hundreddollars a user as long as it does what it needs to do, but that I think thatprovides valuable insight into the business yeah. We need some sum shotsand guess what some of those assumptions didn't lit up and that's:okay, exactly another way that we kind of talk about this internally. Is Weuse the word in Nursia thinking, which is just you start to justify thingsbecause of the way things are like? Oh well, that code exists and there's ahundred lines of it. Therefore, we can't change. It is really important tonot ever do that and always ask, but is this code still relevant today? If youknow maybe it was the right decision to year go to write it, but it might notbe today and if that means that we're going to delete all this code, that'stotally fine. We shouldn't get hung up about that. If it's the right decisionto make for the teacher right now that that makes sense, but that I think thatrequires you now some trust within the organization that Licas we can sit hereand say this thing that we spent you know three months on last year doesn't help her customers any more.Maybe we've evolved past this and...

...that's that's fine exactly you know,that's probably someone's baby or someone's sacred cow right, so yep, sothey have to you, have to kind of get over that emotional attachment andsorry just to like keep pushing on this point further, just minting thisemotional attachment to things. That's also the way that we framedisagreements internally. Is that disagreements or opportunities to learnthey're, not opportunities to be right, and so a disagreement means two peoplehave conflicting assumptions somewhere down the line and it's our job to findthem identify them. You know figure out the right way forward or sorry, Ishouldn't say the right way forward- figure out a way forward that allows usto revisit that decision with no emotional baggage, because if itbecomes an I'm right, your wrong argument, then, regardless of thedecision that's made, you know if you do need to revisit it six months downthe line. Someone seen like, I told you so and that doesn't help anyone whenyou are a thinking like that, were there any resources that help form someof this, you know I'd call it likeorganizational psychology that you guys are implementing and you know helpingyour team embrace. You know: was there a book or maybe a podcast or leader outthere, that you know helped make this Sasin? Maybe it's a collection ofthings I'm interested in this, because this is what I do is I build teams, and you know, structure around that and howdo we work? And what do we do? That's what I've spent the last fifteen yearsdoing. So it's it's really interesting, because you have a very mature outlook on growth, which is not commonin lots of organizations big or small yeah,and I appreciate that. I think I think there are some like I'll, say,intellectual seeds and then at least one book that we do give people so,intellectually, I think, comes from a few places. I mean Jeremy and Iactually are both. So I have a Ph and Matt. He comes from risk and so veryprobabilisic thinkers and I think that's one aspect of it. So that's notabout being right. It's about preserving optional, ty and someprobabilistic sense and then working through that. That's a little bitharder. I think, to to get to certain folks, because you know there's athere's a lot there. Another one just for pure engineers is actually justprinciples of software design. So I actually look up a lot to sandy metshe's a ruby programmer, and she has some really great books on on designthat I encourage all the engineers to read and there the philosophy behindthem is that there's always change happening, and so your job is to solvean immediate need, while preserving the ability to change and unknown ways inthe future, and so engineers really can get their hands on that. And you know:They've lived through a code base for a year and know what that feels like andthen all you have to do is say and the same is Tue at the organizational leveland the decisions and processes we made they're always changing. We shouldnever optimize for a static state of the world because that doesn't exist,there's no equilibrium, especially at a hydro start up and then. Lastly,probably the most concrete answer is creativity ink? Actually, he picks ourbooks. We give to every new employee and in there are a lot of elements ofthis. So, just the idea that every idea starts bad and you have to work toforge that idea and- and you know, revisit things and throw things awaywhen they're not working, very cool love. That yeah, I mean you knoweverybody's journey's different. You know, I've probably consumed fortybucks on business and building culture, and a lot of the themes are the sameright. It's just how it's how it's presented, or maybe some of the storiesthat you tell with it. So that's really interesting. You know how you guys havearrived to. This is how we want to communicate to each other right, and Ithink you probably read Ben Horror. It is you are you are what you do, I thinkis the title, but I think the the important thing here is that we cantalk all day about the intellectual side of it, but it's really importantto have ways of demonstrating it, and...

...so just one way that we do this onceagain on the engineering side is we have a weekly hour hour and a half longopen? It's not really a meeting but session called product office, ourswhere we just any idea, no matter how insane is totally, you know open for discussion. The idea could belike we should pivot entirely. You could sayanything you want and then we all just talk through it and there's no pressureto reach any conclusions, but it kind of gives us that place to continuallyhave these discussions of how do we make decisions and think about ourproducts so that everyone can start to get a better sense for what we mean? Wetalk about these things in the abstract, so it's a structured way to do audiat,basically exactly exactly. Sometimes we get features out of it, and sometimeswe just you know, talk about finding cool technical challenges, a sure yeah,that's awesome. So one of the things we always try to tackle on the podcast isthe theme of speed, scale and savings. So that's a little sales I'll just goahead and say it, but at the end of the day, when you're, when you're buildinga product for customers, you usually trying to solve for one or all of those things. So maybeit's a two part question here. How do you think, maybe you built your yourapplication stack with that in mind, because you know you're, probably not.You know open sources, one of those decisions right, but you're, probablyleveraging other open source projects. I would imagine to some degree, somaybe how you're doing that internally and then how do you convey that to yourcustomers in a way that resonates with them? And so just to be clear, you areyou referring to speed, scale and savings that the product delivers, ordo you mean speed, skill and savings internally when we're developing theproducts, probably both yeah. So I'd say: Let's start internal, you know,what's some decisions that you made there that you know maybe aligned to some of thoseattributes and then shifting to the customer focus side. You know what ismaybe some of the value prop of your offering the customers that theyattached to. You know what gotch might be speed. You know for your customersright work, flows, kind of important from a from the speed perspective,that's where it's hard, but yeah internally. You know what what aredecisions or inflection points within maybe how you're doing business withyour development team that aligned at one of those three attributes yeah. SoI think, speed and scale or the two that resonate most with me and there'sa few different dimensions to that internally. So first is we want to movequickly while gaining knowledge, and so that kind of goes back to what I wasreferring to about releasing mvs to get quick feedback like we want thatiteration cycle to be one where we're constantly gaining knowledge- and youknow, writing down thoughts about about ways which the product can improve s,and so we do and also responsiveness to especially to our open source communitymembers, and so we have a really really active slack community tons ofquestions today and we try really hard to respond. You know someone to respondwithin thirty minutes of every question being answered and that responsivenesshas gotten us a long way created a lot of good will that we really are here tohelp people and so so speed. In that sense, that's both an internal, and youknow, value ad that we provide on the support side. Savings in terms of whatthe product offers is very largely about both development time andmaintenance time to get saved. So one of our product principles is, youshould be able to. You know, have a minimal set of quicks in the DASH boardto quickly get to at least where you need to focus your attention whensomething is failing, and so that fast fast debug ability cycle is reallyimportant, and I think that most of...

...most of our customer, like case studiesand things, focuses on that engineering time saved a wait. A way to think aboutthis, especially when you think about Defensive Code. DefensiveInfrastructure is especially in the data space peoplewill routinely say things like. I spend ninety percent of my time. Writing whatamounts to defensive code, whether it's data, cleaning or try, accepts logging,etc, and so it prefect can do just the smallest amount, and so, if you thinkabout it in the data space, a lot of data folks will say something onlinethat they spend ninety percent of their time. Writing what amounts to defensivecode so, whether it's data data quality day to cleaning, try, accepts loggingthings like that, and if prefect can get that de number down to eightypercent, so just a small amount. You know, seemingly of time savings we'veactually doubled an engineer's productivity because they went from tenpercent to twenty percent of their time. Writing Innovative Code, and so theproblem space is, is a really powerful one to focus on, and you know we seethat and born out a lot in our case studies and our KBS with customers thattime savings is just the huge part and operationalize things to right, so themore efficient we can be about the way we work allows our other side of ourbrain. That does innovative work to solve bigger problems instead ofconstantly putting out fires exactly exactly- and I think you know, thefuture is continued to focus on getting that negative engineering, defensivecoding down and then eventually your layer on top. A lot of you know insightdelivery about. Oh, you know this task has been slowly taking longer andlonger over time. You know predictive analytics things like that Goshi, so wealways like to wrap up with asking about some advice that you would giveto someone who maybe aspires to do what you're doing or maybe they're in themiddle of it right now and they need some encouragement. You know, what'smaybe a a best practice, good piece of advice, something interesting along theway that you share with people commonly that kind of thing yeah. I have a fewdifferent thoughts, so I mean there's the culture side think about youroperations is sooner than you think you need to etc. I kind of touched on thatI think maybe more interestingly, is especially the founding team or theleadership team I do have. I guess, advice for them for anyonelistening. So the first one is, is someone once told me the worst job tohave it? I start up is the job is CEO wants and for Tech Start Upspecifically that job tends to be owning the product, so something andproduct plan, and all that really means is that in the early days, when you'restill small and having to make really large decisions, you know the foundingteam is going to have a very strong opinion about the direction of theproduct and it's actually important for them to be really involved in a lot ofthose day to days for as long as it's feasible so that you know the directionof the company stays consistent, division stays consistent, etc, andthen, of course, eventually you start to scale that out and that's that'stotally fine, so just staying involved for as long as you can is my firstpiece of advice and then on the flip side of that is you know, I kind ofjust say this: a lot like don't optimize for static state of the world,and so do not think of your role as a set of you know known responsibilities.I think it's your job at a high growth start up to just be as absolutelyadaptable as possible, so that you can unblock your team and you know reallystart to empower them to make innovative decisions to make mistakes.Things like that, and so I've seen people who get maybe nervous becausethey're feeling time pressure, but they also don't want to hire for it becauseit feels like something that becomes you know they. They now are no longerresponsible for it, and that's almost always a good thing, because there's somuch to always do o there's just no way...

...you're going to run out of things to doif you, if in when you have to delegate yeah, absolutely we've been passingaround this this phrase for the last five or sixweeks, and it was how would you solve this problem if youdidn't have a boss to go to? Oh, I like that you know like. Can you figure itout and there's some many smarter way more famous. That said that, but it'spretty clever right, because you either you're either focused on solvingproblems or you're waiting out other people to soldier problems and whenyou're in a Higera software company, you know if you're building a productyou get to solve problems creatively, often without a ton of mentorship. Youknow just try some things out and see what happens right, get guns when youcan, and that goes right back to that empowerment is the ability to makemistakes and not you know, fear the consequences. Yeah, absolutely you knowlike we focus more on consulting so for our customers. We have to be creativeand take some risks and great well you're doing things this way. How aboutwe try this this? This would alone with what we think of best practices, andyour use case is strange, which usually is coding for poorly poorly developed, so we're going to recommend doing thingsthis way and see if that works. You know andthat's you know, consulting scary building a product is scary too,because you know you have people paying for it and if it doesn't work yep, itcan get weird. So exactly well, Chris. This has been super, enlighteningappreciate your time and your your transparency and openness about youknow your journey today very interested to see how you guys continue to rise onthe future yeah thanks again nick. This is a really fun conversation. Thanksfor having me all right, thanks, Chris have a good one. You, too applicationmodernization is sponsored by Red Hat, the world's leading provider ofenterprise, open source solutions, including high performing Linux cloudcontainer and Cubantes technologies. Thanks for listening to application,modernization, a podcast for high growth software companies, don't forget to subscribe to the showon your favorite podcast player, so you never miss an episode and if you useapple podcast, do us a favor and leave a quick rating by tapping the starsjoin us on the next episode to learn more about modernizing yourinfrastructure and applications for growth until next time. I.

In-Stream Audio Search

NEW

Search across all episodes within this podcast

Episodes (13)