Sam Altman - Lecture 11: Company Culture and Building a Team, Part II lyrics

This text is annotated! Click on the highlights to read what others are saying. If you'd like to add your own insights, comments, or questions to specific parts of the lecture, visit the lecture page on Genius, highlight the relevant text, and click the bu*ton that pops up. Your annotation will appear both here and on Genius. Sam Altman: Part two of Culture and Team. We have Ben Silbermann, the founder of Pinterest, and John and Patrick Collison, the founders of Stripe. Founders that have obviously, some of the best thinking about culture and building their teams. There's three areas we are going to cover today. One will be general thoughts on culture as a follow up on the last lecture. And then we are really going to dig in to the findings of these companies and building out the early team. Then how that changes and evolves as these guys have scaled their companies to the hundred plus, I don't even know how many employees you have now but quite a lot, these very large organizations and how they adapt these principles of culture. But to start off, I just want to ask a very open ended question which is, what are the core pieces of culture that you found to be the most important in building out your companies? Ben Silbermann: Sure. What are the most important parts? For us, we think on a few dimensions. One is who we hire, what those people value. Two is what we do every day. Why do we do it? Three is what we choose to communicate and I think four is how we choose to celebrate. Then the converse of this is what you choose to punish, but in general I think running a company based on what we celebrate is more exciting than what we punish. I think, the four things I think make up the bulk of it for us. John Collison: One thing, I think Stripe has placed a large emphasis on, more so than other companies, is transparency internally. I think it's something that's been really valuable for Stripe, and a little bit misunderstood. All the things people talk about, like hiring really great people or giving them a huge amount of leverage. Transparency for us plays into that. We think that if you are aligned on a high-level about what Stripe is doing, if everyone really believes in the mission, that if everyone has really good access to information, and everyone has a good picture of the current state of Stripe, then that gets you a huge amount on the way there in terms of working productively together. And it forgives a lot of the other things that tend to break as you grow a startup. So as we have grown, we started off two people, we're now a hundred seventy people, we've put a lot of thought into the tooling that goes around, transparency. Because with a hundred and seventy people, there's so much information being produced that you can't just consume it all as a fire hose. And so, how we use email, things like that, we can go into it more later. But that's one of the core things that helped us work well. Patrick Collison: I think culture, to some degree is a resolution to a bandwidth problem. In the sense that, when you start out working on something you are coding all the time, but you can't code all the things that you think the product may need. And so the organization gets larger. Maybe in some idealized world, I don't actually think this is true, but ideally you should be involved in every single decision, in every single type of moment of the company, everything that happens, but obviously you can't. Or maybe you can if you're two people, but you certainly can't if you're five or ten. At that time, it comes very quickly, by a hundred and fifty, it's completely hopeless. And so culture is the invariant that you want to maintain, as you get specifically involved in fewer and fewer decisions over time. When you think about it that way, maybe intended importance becomes self-evident. Again, the fraction of things you can be involved in directly, diminishing, exponentially, a**uming your headcount growth is on a curve that looks like one of the great companies. And yeah, that's super important. It manifests itself in a bunch of different ways. For example, in hiring, maybe the reason the first ten people you hire, the decisions are so important that aren't just hiring those first ten people, you are actually hiring a hundred people because you think each one of those people are going to bring along another ten people with them. And thinking exactly what ninety people that you would like those first ten people to bring on. It's going to be quite consequential for your company but really briefly I think it's about abstraction. Sam Altman: One thing a lot of speeches in this cla** have touched on is hiring those first ten employees. If you don't get that right, the company basically never recovers. But no one has talked about how to do that, so what have you looked for when you have hired these initial employees to get the culture of the company right? How have you found them? Ben Silbermann: I guess this answer is different for every company. I'll say for us it was very inductive. Clearly I looked for people that I wanted to work with, and I thought were talented. I have read all these books on culture, because if I don't know something, the first thing I do is go read about it. Everyone has all these frameworks, so I think one big misconception that someone said once was, people think of culture as architecture when it is a lot more like gardening. You plant some seeds, then you pull out weeds when it's not working, and they sort of expand. When we first hired people, we hired people that were more like ourselves. I often looked for three to four things that I really valued in people. I looked for people who worked hard, had high integrity, low ego. I looked for people who were creative, super curious, which meant they had all these interests. Some of our first employees are some of the quirkiest you have ever met. They were engineers but they had all these crazy hobbies. Like one guy made his own board game, with his own elaborate set of rules. Another guy was really into magic tricks, and he not only coded a magic trick on an iPhone but he shot the production video in the preview. And I think that quirkiness is a calling card that we find, the people that are excited about many disciplines and extraordinary at once, tend to build really great products and are really great at collaborating. And the last thing, we really want someone who wants to build something great. And they aren't arrogant about it, they want to take a risk and build something bigger than themselves. And that, in the beginning, is very easy to select for. If you were in our position, we were in this horrible office, nobody got paid. There was no external reason to stay except wanting to build something to join. In fact there was every reason not to. And that's something, looking back, that I really value. Because you always knew people were joining for the purest of reasons and in fact forgoing other job opportunities, market salary, a clean office, good equipment just for the chance to work here. To this day, I think a lot of those traits are seeded and embedded in the folks that we look at now. John Collison: The first ten hires is really hard, because you're making these first ten hires at a point where no one's heard of this company before. Nobody wants to work with you. You are these two weird people working on this weird idea- Patrick Collison: Their friends are telling them not to join. For our second employee, he either accepted the offer or was just about to, and his best friends took him out the night before, it was a full on a**ault, why you should not join this company. Why it is ruining your life basically. And the guy continued to join, actually one of those friends now works at Stripe, but this is what you are up against. John Collison: And it's hard, no batch of ten people will have as big of an influence on the company as those first ten people. And I think that everyone's impression of recruiting is you open LinkedIn and it's sort of like ordering off the dollar menu, I want that one, that one, that one and now you have some hires. At least for us, it was over a very long time period talking to people we knew, or friends of friends into joining. We didn't have huge networks, we were both still in college by then. So there were really no people that we worked with to draw in. So a lot of those early Stripers were people we heard of from friends. The other interesting thing they had in common, they were all really early in their career or undervalued in some way. Think about it, if someone is a known spectacular quantity, then they are probably working in a job and very happy about it. So we had to try to find people who were, in the case of our designer that we hired, he was eighteen and in high school and in Sweden at the time. As the case with our CTO, he was in college at the time. A lot of these people, they were early on in their careers, and the only way we could, you can relax one constraint, you can relax the fact that they are talented, or relax the fact that it's apparent that they are talented. And we, not consciously, we relaxed the latter. Patrick Collison: Finding the right people, you have to think like a value investor right, you're looking for the human capital that's significantly valued by the market. You probably shouldn't look to hire your friends from Facebook, and Google, or whatever, they are already discovered, and if they want to join you, that's great. They are probably harder to convince. John has spent a little while yesterday afternoon trying to figure out in retrospect, what traits out first ten people had in common and felt were significant. Generally speaking in culture I want to carry out everything we say, advice is very little advice, wildly extrapolated and I think there is a lot of truth to that. For our first ten people, the things that seemed to be important, they were also very genuine and straight. And I think that matters quite a lot, there are people that others want to work with. That there are others that people trust, that they are intellectually honest on how they approach problems. They are generally people who like to get things finished. There are a lot of people who are really excited about tons of things. Only some of those are excited about completing things. There is a lot of talk out there, like hiring people off their GitHub resumes, that doesn't really ring correct to me as there is a large premium on lots of different things. I think it's much more a priori, much more interesting to work with someone who took two years to spend time going deeper into an area. And then the third trait that we looked for is that they cared a great deal, it's offensive to them when something is just a little off. In hindsight there were all these crazy things that we use to do that, do in fact, seem crazy. Like I should have not done them. Everyone was always like, it was borderline insane how much they cared about tiny details like we used to. Every single API request that ever generated an error went to all of our inboxes and phoned all of us. Because it seemed terrible to get an error that didn't get a resolution from the users standpoint. Or we used to copy everyone else on outgoing email and point out slight grammar or spelling mistakes to each other. Because it would be horrible to ever send an email with a spelling mistake. Anyways, those are the three traits we came up with, genuine, caring a great deal, and completing things. Ben Silbermann: I have something to say, I don't think there is a wrong place to find people. So when I look back at our first hires we hired, they came from all over the place. I put up ads on craigslist, I went to random Techtalks, we used to throw weekly BBQ's at the office, bring your own food and drinks and then we would just talk to folks. I think every time I went to get coffee at Philz, one of you guys were recruiting. Because your office was strategically placed next to the best coffee shop. But I think the really good people, generally are doing something else so you have to go seek them out instead of expecting that they are going to seek you out. Triple when no one has ever heard of or is using the product that you work on. John Collison: Yeah and it's probably really important to have a great elevator pitch, not just for investors but because everyone that you run into right now is six months to a year down the road a recruit. So the right time to have gotten them excited about your product, the right time for them to have started following us, is as soon as it can start. It's going to take a very long time to recruit people, so getting people consistently excited about what you are doing will pay back later. Patrick Collison: Maybe this is a little tangential, but a bunch of our friends started companies right out of school. And we started thinking, what goes wrong in those companies? And I think the most common failure mode was doing something overly niche, overly specific or bad. I think there is a major shift in time horizon as you go from cla**es to a startup. A cla** plays out on a quarter, or a semester, where a startup is a five or ten year thing. And I think this is really problematic, because it's really quite hard to hire people for niche things. If you told someone, look we are going to build a rocket that goes Mars, that sounds almost impossible but sounds f**ing awesome. It's really easy to convince people to work on it. Instead of, well we are going to work on, I'm not going to give any specific idea, probably going to sound like we are doing a startup. But if you pick something pretty narrow, generally comes out of this cla** project that's actually much harder to hire for. Sam Altman: One specific specific question that has come up a lot is how, as a relatively inexperienced founder, you identify who the really good people are. So you meet people at these BBQ's, you are friends, maybe you have worked with them. What did you guys do specifically to identify that this person was going to be really great? Or did you really get it wrong? When did you learn you could identify raw talent? Or say they work at Google, or Facebook so they must be good. Ben Silbermann: You will never 100% know until you work with folks. So the flip side is, if the person you hired is not a good fit, you owe it to them and to the company to tell them where to improve and if they aren't working out, then to fire them. But I think, generally, the question of talent falls into two big buckets. One is, you have some sense of what makes them good at their job. And there are some areas where you can test that area. And there are some that you don't. And the ones where you don't are much more difficult. So what we would do is a few things. Before we talk to anyone, we try to figure out what exactly is world cla** in that discipline need. And this comes in a little later when you are hiring the head of finance and you know nothing about finance, except what is contained in a library book that you got. Like an introduction to finance or marketing. So I always made it a habit of mine to talk to people I knew de facto were world cla** and just asking them, what are the traits you look for? What are the questions you ask? And how to find them? If you are looking for the next person that is as good as you, where is that person working right now and what's her phone number? I think that learning what's good and bad during the interview process is extremely expensive. It is an expensive use of your time, expensive use of everyone else's time. A recalibration of that really matters. Then once you have someone in an interview process, you will build the process over time to screen quality. Pinterest, we have an evolving set of questions that are rotating through and we are always asking if these are good indicators or bad indicators of quality. The other thing the questions are supposed to do is give us a sense of, is this the right place for this person to come in and work? This is the point you guys made about being very transparent. About what's going to be easy or hard. Really great people want to do things that are going to be hard. They want to solve tough problems, so there was a sense of brilliance in Google sending out these interview questions that were thought to be really difficult. Then people who like solving problems, they come out and seek those. I think it's really important as companies get bigger that they don't whitewash the risks. I heard that Paypal, you go in and after the interview they say, by the way, Mastercard wants to k** us and you will be doing something that is illegal, but if you succeed you will redefine payments. Or when they were recruiting for iPhone, they didn't even tell people what they were doing. You won't see your families for three years, but when you are done, your kids, your kids' kids will remember what you built. I think that's a really good thing in recruiting as well. Be very very transparent on why you think it's a great idea, but you lay out in gory detail why it's going to be hard. And then the right people select in or they select out of that opportunity. Patrick Collison: Evidence suggests they were able to see their kids though. John Collison: I think one thing you have to do when identifying talent is have the confidence to interview in a way that works for you. I think, say you are not the world's best engineer and you are trying to interview engineering candidates. I thinks it's tempting to co-opt what everyone else does, get them to put things on a whiteboard and do other engineering things. In the case of Stripe, we flew a guy out and we spent a weekend coding with him and looking over his shoulder. It was the only way we could tell and get ourselves confident if that guy was any good. And I think you can extend that to any roles you are not an expert in. In that I am no business development guru but when we hire for business development roles, we have a project that we have them talk about, how they would improve an existing project that Stripe has or which new projects they would go out and do. And even if it is not my domain area, I am confident enough I can judge those really well. I think people often have this imposter syndrome when it comes to interviewing for roles. Patrick Collison: I think a specific tactical thing to do, again, for the first ten people is to work with them as much as you can before committing to hire them. Once you hit a certain scale it's kind of impractical to put them on that side and be unsk**ed. Expensive from your side. But it's really worth it to the first ten people, right. In the majority, the first ten people, we worked with them to some capacity for a week in advance. It's pretty hard to fake it for a week, it tends to be quite clear quickly. Another answer I thought of to the question, how do you know if someone if great? And people talk about this notion of the 10x person or what the sk** set is. I don't know what 10x means. I think the slightly more intuitive decision is, is this person the best out of all of their friends at what they do? It's a little insensitive on how they choose among friends, but for me at least I find that a better way to think about it is, is this the best engineer this engineer knows? And the other thing worth mentioning is, on the first ten people on the culture and team topic, I think everyone doesn't realize until they go through it themselves, how important it is because in life and media people focus too much on founders. Here we are and we are reinforcing the structural narrative that Stripe is about John and Patrick and Pinterest is about Ben. When the vast majority of what our companies do, 99% are done by people that are not us, right? It's obvious when you say it but it's very much not the macro narrative. These are abstracts and you a**ociate them with certain people. For companies like Apple and others, Steve Jobs was a tiny, tiny part at the end. John Collison: So don't screw up is what you are saying? Patrick Collison: Something like that. Ben Silbermann: I think referencing people is really important Referencing people is just what it sounds like. Asking people with experience for their honest opinion. We do that really aggressively but we are trying to figure out what this person is like to work with. We are trying not to validate if they told the truth on their resumes because we a**ume they told the truth. So a very standard question is in an interview I might say, hey we both know Jonathan, because we are both friends, if I asked him what you are best at, what you are the most proud of, or what you were working to improve, what would he or she say? Because it creates social awareness and accountability. Then I typically ask something that makes the question, that is typically soft, feel a little bit more quantitative and then calibrate that over time. To evaluate this person's dimensions, is this person the top 1% of the people you worked with, the top 5%, and the top 10%? And it forces scarcity that gives them material reference. Instead of saying, hey what's the best thing about John? You say, he told me he was good at these things. Can you validate it? Yeah, sure. It's the type of tool that should be taken seriously. John Collison: And referencing, obviously, isn't easy to begin with. But it does provide really useful over time. And I think for name references people do want to be nice so you have to create an artificial scarcity by saying, where would you rank this person with people you worked with. You should aim to spend fifteen minutes on the phone with that person instead of letting them say, yeah this person is awesome. Sam Altman: Also those references are a tremendous source of recruits. Once you hire these first people and they join, what have you done to make them effective quickly and to get them to the right culture place? Hiring is usually difficult but then not as difficult as making them happy and effective. So what do you do with these early employees to accomplish that? Ben Silbermann: Well the answer to that has changed since we went from small to bigger. When we first started it was because we needed that person, a long time ago. So their whole onboarding was, here's your computer, we already set up your environment, don't worry about it, this is the problem we have to solve together. That's the nature of the startups, we were all in this tiny two bedroom apartment. All the other things, building personal relationships, spending time together, it all happened magically. You didn't have to do anything. The one thing I would like to add to that is we always reminded people where we want to go with that someday. Because it's really easy to drop someone into a problem and they would think the whole world was this little problem in front of them. We would always say, someday we want to do for Google what they did for search. Our plan for trying to get that done. Now as the company grows, I think that problem has to get a little more formalized. So we spend a lot of time thinking and constantly trying to refine what that person looked like from the day they came in, to their first interview, through 30 days after they joined. Do they have someone's name they know? Do they know who their manager is? Have they sat down people on their team? Do they know what the general arch of the company is? And what the top priorities are? And we have a program we do. It's a week long and they have functions to go in deeper. And that's something that has always been refined. And the output metrics on that is one, we ask people, what did you think afterwards, then 30 days afterwards? Then we also ask their peers and manager, hey is this person up to speed? Do you feel we did a good job at making them productive? If we haven't then thats a key that a) we should not be hiring any more people because we're not doing a good job bringing in new people and b) we need to retool that. I think those things are important. I just wouldn't discount how important it is to get to know the person as a person. What's their aspirations? What's their working style? How do they like to be recognized? Do they really prefer being in total silence? Are they a morning person or night person? Knowing those things, it just demonstrates that you care about them individually and collectively, what your goals are. John Collison: I think there are two things that are important at any stage, though the implantation will change. First is to get them up and running quickly to do the work. That is how you are going to find the problems, it is how progress is measured in the real work they are doing. And so when we have engineers start, we try to get them committing on the first day. When we have people in business roles start, we will have them in real meetings the first day on what they are meant to be working on. Sometimes it's easy to be tentative and ease people in. We are much more, push people off the cliff. Then second, we try to quickly give people feedback. Expectedly giving people feedback on how to adapt to the culture. When you think about it, if you have built a strong culture as all the companies up here are trying to, it's going to take some adapting from the person, it's not going to be necessarily easy. One thing we have at Stripe is the culture is a lot more written. So you have people next to each other, with headphones on, IMing each other. And for a lot of people coming in and working in an environment like that it's sort of hard- Patrick Collison: In from normal places. John Collison: Exactly yeah. So from everything high level of how you are doing at your job to minor cultural issues, the more feedback you give them, the better they will do. Its unnatural to be telling people if they are doing a good or bad job. You don't do that in your normal life, hopefully you are restrained. But when you have employees that is what you owe them to do well. Sam Altman: So I think this is a good transition to when you're companies have scaled. What are the biggest changes you have had to make to your hiring policies and to how you manage the teams as you have gone from two to ten to a thousand employees? Ben Silbermann: There are a lot of changes. I think one thing we try to do on the team side is to make the teams feel as autonomous and nimble as possible within the constraints of the organization. That means over time we are trying to make it feel like a startup of many startups. Rather than this model setup with form policies cut horizontally through it. It's easier said than done, I don't think we are all the way there but one goal is that each team has control, to hold the resources that they need to get the goal done. They know what the most important thing is and how it's measured. That way the management problem because somewhat tractable. Otherwise it feels completely impossible if you can't decompose it into atomic components. You just look at it like, Oh my gosh, complexity level is rising geometrically, on a management level, it's never going to work. You have to create these abstracted units. At least that's what we are going to try to do. Pinterest in particular, the real challenge with building those abstract units is, we want units that encompa** a super strong designer, or a super strong lead engineering, a writer, often times a community leader. We want them to be self contained. That kind of makes it hard, but that kind of core is to our philosophy to build products. We put people together that have all these kind of disciplines, lots of things then we anchor them to a certain project then we try to remove barriers to let them go fast. Then we find no barriers and we sit down to figure out how we speed that up. I think hiring is a little different. I think the biggest change and the biggest a**et you get is the people, referrals become more and more the life blood depending on the network of the people you bring in. So one of the lucky and in hindsight decisions we made was our actual fourteenth or fifteenth person we hired was a professional recruiter. She worked at startups, she worked at big companies like Apple. But she sort of knew where that pipeline breaks down. Knew the early indicators, and taught everyone not just how to screen for talent but to identify the people who are going to be culturally really good for the company. And I think looking back on that, its something I personally really value. Patrick Collison: There is a huge amount of stuff here that is under management growth. Either your company fails really quickly or all of your problems become about management growth. One thing that tends to take people by surprise, and took me by surprise, is how quickly the time horizons change. In your first month, you are largely thinking about things one month ahead, right? Maybe that is what your development road map is oriented around. Who you are working with, maybe it's a very informal relationships where they haven't fully committed to be full time or not. The more time that goes by, I feel that has a reciprocal in the time horizons. In one year, you are thinking a year ahead. After four years you are thinking four years ahead. That increases very quickly right? After one month it's super short term. After eleven months, you should actually be thinking and planning a year ahead and think about human structures. Thinking about stuff Ben talked about. Where you want to be going long term. Things like that. I think that plays into the hiring and that in the early days you have to hire people who are going to be productive. Essentially , you don't have the luxury of hiring people that look to be promising but they are not going to be up to speed for another year or two. They have to be able to work immediately. But after two or three years, then it becomes much more reasonable to make those investments. Actually if you are not making those investments you are probably being much too short term. And so I think that's really important. All of these problems in some sense are easy. Like, how do you build such good bonds with people? We all do it every day. How do you make it systematic, and effective at scale? It's all going to vary significantly, in perfect proximation of what you would ideally do small, what hacks can you pull? Make it work as well as you possibly can at a large scale. A rapidly growing company, with head count here two-three heads a year, it's a very unnatural thing. What's the least bad way of managing that growth? I think it's worth being systematic about it, thinking of ways to do it. Realizing you can't do much better than ask a question. For Stripe, it's things like, we have three meals a day at long tables where everyone can sit together, if you think about how much more human interaction happens with having these randomly mixed meals. It's vast, right? A whole list of things like that. I think that's the general framework. Ben Silbermann: One thing I'm really curious about. You guys value transparency. Have you scaled it over time? I know with us, we think about it all the time. Just curious. Patrick Collison: Startups, I can't remember who defined it as, a startup is an organization that is not yet stuck with all these principal agent problems. That large companies, what is locally optimal for you frequently is not what is globally optimal for the company. As a consequence of that, a reason a startup can just work differently than a big company, at a big company, a lot of the things that are good for you, you couldn't do them in a completely transparent environment because people would think less of you. But because everyone is rolling in the same direction, a startup, you can kind of make all the information transparent. Like I said earlier, Stripe used to bcc us to be on every email unless you opted out of it. We thought that would be more efficient, you wouldn't need to have as many meetings if you could keep abreast of what is happening. And over time, we sort of built an interesting framework of mailing lists. We now have a program for generating gmail filters. For a rocky path of fifty people or so to Ben's point of asking people how they are getting along after several days. They all reported terrible because they couldn't find all the emails people were sending to them. They were missing things and everything. John Collison: Gmail broke at one stage. Patrick Collison: Right, right. At one point Gmail broke because we were sending too much email. It is hard to scale because you might contact somebody out of the company with some great idea. The person sitting across the way from you thinks it's the stupidest thing they have ever heard. You are kind of under that scrutiny of the whole organization in some degree with all your communication. That's the challenging side, that people more formally know what's happening. I don't feel I can give a stronger endorsement of it than it has worked so far. I really am curious how it will work when we are five thousand people, if we are ever at that scale. John Collison: I think a couple things that helped us scale it, is we changed tools in changing the culture around it. On the tools front it use to be the case you could keep abreast of what's happening in the company by reading all the email. Now we weekly all hands on deck. We actually have to put all this work into communicating what is going on in this company. Since there is so much more. The second is, cultural side so much information in terms. You have to create social norms around it. Obvious things such as what is confidential to Stripe. Less obvious things, like when emailing someone or talking in Slack or IRC that is now viewed by one hundred seventy people, it's pretty easy to get stage fright. And it's pretty easy with what you thought was a reasonable proposal, you get this drive by criticism and you are less likely to share in the future. We have had to create norms around when it is okay to jump into discussions and how that interaction works because people are around the stage much more. Patrick Collison: I'm sure it's not good to put anyone on the spot, but Emily interned at Stripe this summer. I am curious, as an intern. what you thought of it. Emily: Overall it's great. The first week, I spent most of my time reading Hackpad trying to get caught up on what the company was doing. It can often be quite distracting from your own work as often times there are other parts of the company that you are interested in. Patrick Collison: Hackpad, by the way, is like Google Docs but with a news feed, where you can see all the documents. Emily: And you are encouraged to make everything public, everything that you work on. But overall it gets you sped up rather quickly. There are these things called Spin Ups, where every leader of the team gives a thirty minute talk on what they are doing and how you can contribute if you are interested. Patrick Collison: Do you think transparency was that good? Emily: Yes, I remember having a hard time remembering what I should and should not subscribe to. The first week having two thousand emails in my inbox, then by the end there are three or four teams you want that information coming up from. Sam Altman: So this question is for Patrick and John. Have the people that you hired early been able to grow up into leadership roles? John Collison: In Stripe's case yes. A lot of the first ten people are in leadership roles now. I think that's one thing that corporations, it's an unnatural sk** that they need to get good at. People don't exactly come out of the womb being good at management or at leadership. And being able to develop that in people and helping people progress as they spend a number of years at the company, it's a lot of work when people are running around with their hair on fire. But it's also damaging if the company can't develop that sk**. Ben Silbermann: I think for us the answer is some yes and some no. I think one of the benefits of working at a startup is you can be handed a challenge no one else would be crazy enough to let you take on. And that could be managing people, it could be taking on a project. And also if you ask someone to take a risk like that, it shouldn't be one way through the door if you don't succeed. Otherwise it creates fear to give it a shot. So we have some folks managing a large team at the start, individual programmers, individual engineers. And they say, Hey I would love to try leading a project, leading a group, then taking responsibility for management. Then we have other folks that try it and are really glad they did so they know they never want to do that again. We try that, for those people, you can have just as much impact on the company through your individual contributions as an engineer or what have you. But it's really hard to predict unless you give people a shot. So my strong preference is you give as many people a shot as possible. And in the few areas where you feel there is too much of a learning curve relative to the business development you are trying to achieve, that is when you look for someone who might walk in and really execute well on the job. So the question was, has the vision changed since we initially started? Well I think on the vision, when we first started hiring, we were like we are going to build this really cool tool, people are going to enjoy it. I like collecting things, maybe others like collecting things. And what we didn't expect that revealed itself early on, was that looking at other peoples collections was, it turned out to be this really amazing way of discovering things you didn't know you were looking for. It becomes a solution to the problem that a lot of other technologies don't have. Over the last year we have poured a lot of resources into building our recommendation products, search products, feed products. Leveraging the unique data that we have, which are these pins that were all picked by someone and hand categorized. And the on the audience side, I think the first big surprise was truthfully when we first started, we didn't know if anyone would really use it. eE were just happy that anybody who wasn't related to us or obligated would use it. The biggest surprise has been the diversity of people and how diverse those groups have been. And I think that's been one of the things that's the most exciting. And the funny thing is, often as the company goes further along your aspirations get bigger. There is this gap that exists, and I tell my team, between where we are and we should be. Objectively we are further along, I feel the gap has widened. But I think that's a really common trait among people who found companies. Sam Altman: So the question is, most start ups are not the iPhone. You can't guarantee that most people's grandchildren are going to remember this because most startups fail. How do you convince people to make sacrifices to do join a startup? Patrick Collison: I think part of why it resonates with people is because it's not guaranteed. If it was it would be boring. There is the prospect of affecting this outcome, but nothing more than that potential. As far as not seeing their families and kids, startups do involved longer hours in the beginning but I think that story is overstated. Even the startups that in the earlier days had some sort of longer working days, have a tendency to exaggerate. It's kind of like the startup version of fishing. Every startup thinks they worked more insane hours than the next one back in the early days. It's like we literally never slept for two years. I think realistically for most people, it's not that big of a sacrifice. I think on average, people work on average two hours more a day. It is a sacrifice but it is not forgoing all pleasure and enjoyment for the next half decade. Ben Silbermann: Even the iPhone wasn't the iPhone before it got done. No smart person you are hiring thinks you have a crystal ball into the future that only you have and that joining is a guaranteed thing. And in fact if you are telling them that and they select in, maybe you shouldn't be hiring them because they didn't pa** a basic intelligence test of certainty and the future. But I think it's fair to say what's exciting and where you think you can go. And where it's going to be hard and chart your best plan. And then tell them why their role in it can be instrumental because it is. I really liked what you said, if you tell people, Hey we are going to go to Mars, it attracts the best people and you are closer to Mars and they know that. What I would discourage is whitewashing all that. And if people are joining because they want all the certainty of Google and the perk of working in a small startup with more email transparency, then that's a really negative sign. For example, when I interview people, they often say, I'm really pa**ionate about what you are doing. I often ask where else they are interviewing. If they list seven companies that have nothing to do with each other, except they are at the same stage, I love the stage of discovery, so I'm interviewing at Stripe, Jawbone, Airbnb, Uber, I'm also putting my resume into Google X, that's a sign they are probably not being authentic, which you care about. And those folks, when things get hard, they won't stick it out and work through it, because they were really signing up for an experience, not for achieving a goal. Patrick Collison: I think the other thing that motivates people a great deal is the prospect of affecting some outcome, is just the personal development angle. And a startup because it's more lightly staffed it's much less forgiving. Whether or not you are the best or the worst person in the world, you are probably not going to alter Google's trajectory. Whereas if you really want to benchmark yourself and see how much of a contribution and impact you can make, the startup is a much better place to test that. Sam Altman: How does your user base affect your hiring strategy? Ben Silbermann: Conventional base, you only hire people who use your product religiously everyday. And that probably works well if you make an API. For us, we screen for people who have vision and discovery online. And they have to know how our service works, and they have to have used it. But they may not be a lifelong user. And for us that's great, we can ask what is the barrier that is preventing you from using it? Come join, we will move that barrier. Help us get closer to that vision. If you read a startup book, there is all this wisdom, but it is only useful if it works in your certain circumstance. So for us, we have had to broaden the lens a little bit and bring in people who are excited about the mission, that care about our product and our approach to building products. Even if from day one they weren't our earliest users. John Collison: The one thing I want to tack onto this is, we touched on it being hard to hire early on for those first employees, you have people with other options, you are very much at the ugly duckling stage. Hiring people who are pa**ionate about your product is a great way to find people. You have a natural advantage over other companies. I know in Stripe's case, we hired four Stripe users, people who we probably couldn't have gotten otherwise. I'm sure it was the same in Pinterest's case where you will get all this benefit at working with Pinterest, like, Hey it's Pinterest. Sam Altman: Thank you guys very much for coming in today.