yego.me
💡 Stop wasting time. Read Youtube instead of watch. Download Chrome Extension

Now Anyone Can Code: How AI Agents Can Build Your Whole App


25m read
·Nov 3, 2024

1984 the Mac brought personal computing to the masses. 2024 we have personal software. You actually are going to be able to orchestrate this giant army of agents. I think of Mickey Mouse and Fantasia, just like, you know, like learning this new magical sort of ability. Suddenly all the brooms are walking and talking and dancing, and it's this incredible menagerie of being able to build whatever the heck you want, whenever you want.

Someone who had an idea for 15 years but didn't have the tools to build it and was able to build it in 15 minutes, and he recorded his reaction. I almost shed a tear on [Music] that. Welcome back to another episode of the Light Cone. I'm Gary. This is Jared, Harge, and Diana, and collectively we funded companies worth hundreds of billions of dollars, right at the beginning, just a few people with an idea.

Today we have one of our best alumni to show off what he just launched, Repet Agent. Amjad, thanks so much for joining us today.

Amjad: My pleasure, thank you for having me.

Yeah, so we just launched this product. It is in Early Access, meaning it's barely beta software. But people got really excited about it. It works some of the time, so there's a lot of bugs, but we're going to do a live demo here. I wanted to like build an app, like a personal app that could track my morning mood correlated with like what I've done the previous day.

So I want an app to log my mood in the morning and also things I've done the previous day, such as the last time I had coffee or if I had alcohol and if I exercised that day. That’ll send it to the agent. Now you have this like chat interface, so you can see the agent just read the message, and it's now thinking.

So what we're looking at here is actually how you might chat with another user, or is this like specifically...

Amjad: Yeah, I mean it's similar. It's very similar to like a multiplayer experience on Rep.

Got it. So here it's saying I created a plan for you to log your daily mood. The app will show your mood, coffee, alcohol consumption, and exercise. It also suggests other features, so for example, suggesting visualization, and that sounds good. Reminders, I don't know, I’ll remember, so let's just go with these two steps.

I think what was also cool is it picked the tech stack that's very quick to get started. So Flask, vanilla JS, PostgreSQL, like very, very good.

So now we're looking at what we're calling the progress pane. The progress pane is, you can see what the AI is doing. Right now it's installing packages. It actually wrote a lot of code. It looks like it built a database connection, all of that, and it's now installing packages. We should be able to see a result pretty soon.

This is really cool because I think a lot of times for new software engineers, one of the annoying parts is just getting all the packages and dependencies and picking the right stuff, and this just does it for you, the agent.

So here we have our Mood app. I can kind of put that I’m feeling pretty good today. I did have coffee yesterday, but I didn’t exercise. Log my mood, go to history.

Jared: So it built a complete web app with just a prompt, like no further instruction from you?

Amjad: Yes, and it has a backend, it has PostgreSQL, and I can just deploy this. So this is already pretty useful. You have this rating, and you have the history, and it's asking me if it did the right thing.

Gary: Oh, it actually is asking you to test it for them?

Amjad: Yeah, it actually did some testing on its own, so it took a screenshot here. It knows that at least something is presented, but it wants someone to actually go in and do a little bit of QA.

Is it using like computer vision to look at the screenshot?

Amjad: Okay, yeah, yeah, and now all the models are multimodal, and so it's fairly straightforward. What's on the back end right now? We have actually a few models because, you know, it's a multi-agent system, and we found different models work for different types of agents. The main model one is Claude Sonnet 3.5, which is like just unbeatable on code. It is like the best thing, but we use GPT-4 in some cases. There are also some in-house models, like we built the embedding model, it's a super fast embedding model, a binary embedding model, and the retrieval system and indexing.

This is all built in-house, and a big part of what makes this work is the sort of retrieval system because figuring out what to edit turns out is the most important thing for making these agents work. You're going a step beyond just RAG, because RAG hits the limit for this, and you basically have to find a new way to search and find the right places to edit in the code.

Gary: Yes, which is actually something that I don't think has happened yet but I think is going to happen. For all these agent systems, people are going to move away from RAG and start building custom orchestration like this.

So this is very notable. This is like a very cool thing that you figured out.

Amjad: Yeah, if just throwing the codebase in RAG is not going to work, you actually have several different representations that allow the agents to do better work.

That's right, and we have the trends thing working right now, nice. So we have a couple graphs. We don't have a lot of entries here. I can actually ask it to create data.

Gary: Oh really, you can have it create data as well?

Amjad: Yes, now it is asking me to deploy because it's done. It's like it's time to deploy. And here we have the activity trends, like how many... what am I doing by day?

Gary: There you have it, it's going directly from just an idea to a deployed web app that anyone in the world can access right now.

Amjad: Exactly. And one of the things I'm really excited about is like this idea of personal software. In 1984, the Mac brought like personal computing to the masses. In 2024, we have personal software. I think we just experienced this. You know, Karpathy just tweeted about Repet Agent. He said, "This is a feel the AGI moment." Did you just feel the AGI?

Gary: I definitely did. And I did last night. I spent a few hours last night using Repet Agent to make a Hacker News clone. Nice. There were a couple moments where like I really felt the AGI.

The first was it actually had like really good intuition about what you buy to make and how to design it. Like we saw that there, like you didn't give it the idea to make the slider bar be like emojis, it just came up with that on its own.

Then the second thing was when I was using it, it really felt like I had a development partner. He would ask me questions. He would ask me to change things. At one point it got stuck, and I wasn't sure how to do something, and so it asked me how to do the thing. I told it, and then it's like cool, got it, and just kept going.

Amjad: Yeah, it feels great. And sometimes you want to give it some help.

Gary: Right, you want to go debug if you know how to debug yourself, or you go ask, chat about something and come back to it. Just give it more information, it'll be able to kind of react to it.

You should have it definitely feels like talking to like a developer. You should do like the gro thing and have different modes. You could have like a "Gry Programmer" where it just tells you ideas are bad and he wants to build something else anyway.

Amjad: Oh, that would be cool, just like have a toggle. For example, like an over-engineer, just like over-engineer everything.

So it added this toggle, but I don't think it works. I don't think it connected up to the x-axis. I think this is interesting about all these AI programmers, which is that it's not like we created some super intelligence that somehow can just build an entire app perfectly from start to finish without making any mistakes.

It actually codes the way a human does, which is it like writes some code and it's like, well, I think this is right, but I'm not sure, I guess I'll try. And then it tries, like, oh no, I have a bug. It's like it's the same thing.

Gary: Yeah, and again our design decision has been always like this is a coworker. You can just close this and you can go to the code and you can code yourself, just fix it yourself. And again if you don't know how to code, my hope is as you are reading what the agent is doing, is that you've learned a little bit of coding along the way.

And by the way, this is how I think our generation learned how to code, not through agents but almost by doing these incremental small things like editing your Myspace page or doing a GeoCities thing. I feel like we sort of lost that incremental learning scale where now you need to go and get a computer science degree or go to coding boot camp to figure this out.

But if we made this like fun thing that people can go build side projects in and get exposed to what code is, I think that would be perfect. And again, my view is that we're still far from fully automated software engineering agents, and people should still learn how to code.

You have to do way less coding, but you will have to read the code. You will have to debug it. In some cases, the agent will get you fairly far, but sometimes it'll get stuck and you need to go into the code and figure it out.

Jared: Yeah, I think that that's actually pretty important. I'm I've been meeting a lot of, you know, 18, 19-year-olds who are freshmen, and they're like, well, the code will write itself, right? Like have to study this stuff anymore.

And I'm like, no, that's not true at all. Like, I actually think that now it is actually more leverage. It is far more leverage to know how to code than ever before, and it's actually even more important, and it will make you way more powerful.

Like you don't have to be all the way in the weeds on everything. You actually are going to be able to orchestrate this giant army of agents. And I think of Mickey Mouse and Fantasia, just like, you know, like learning this new magical sort of ability.

And like, you know, suddenly all the brooms are like, you know, walking and talking and dancing. It's this incredible menagerie of being able to build whatever the heck you want, whenever you want, just like literally from any computer, from any web browser.

Amjad: Yeah, I try to come up with a like a Moore's Law type thing where it's like the return on learning a code is like doubling every six months or something like that. So learning code a little bit is in, you know, 2020, you know, was not that useful because you would still, you will you get blocked? You wouldn't how to deploy something, you wouldn't know how to configure something.

Let's go to 2023 with chat learned to code just a little bit will get you fairly far because chat can help you. And then 2024, learn to code a little bit is a massive leverage because we have agents like this and others, and there's a lot of really cool tools out there, like cursor and others that will get you super far by just like having a little bit of coding and just extend that forward.

Like six months later, you're going to have even more power. So programmers are just on this massive trajectory of increased power.

Gary: Can you tell us more about the tech behind this? It's kind of fascinating.

Amjad: At the heart of it, it is sort of this, as I described before, it's a multi-agent system. You have this core sort of React-like loop. So React is a, you know, an agent chain-of-thought type prompting that's been around for a couple of years now, and most agents are built on that.

But ours is also a multi-sort of agent system. We give it a ton of tools using tool calling, and those tools are the same tools again that are exposed to people. By the way, you need to be really careful about how to expose these tools and how does the agent see them.

So for example, our edit tool returns errors from the language server. We have a language server here, a Python language server, like a human coding. You know, if I make a mistake anywhere here, it will show me.

Similarly, when the agent is coding, it gets feedback from the language server. So again, you want to treat it as much as you can like a real user. So for any action, it gets sort of feedback, and then it can react to that feedback.

And so these are the tools again. This is package management, editing, deployment, all the database, all those are tools. And then there are a lot of things that make sure that it, you know, doesn't go totally off the rails because it's very easy. We've all, you know, used agents that go off the rails and go into endless loops.

This still sometimes does it, but we have another loop that is doing a reflection that is always thinking, am I doing the right thing? We use a lot of L chain tools. So Lang graph is an interesting new tool from Lang chain that allows you to build agent DAGs very nicely, and they have some logging mechanism.

And a tool called LSMITH where you can look at the tracers. Looking at the traces for DAGs is very, very, very difficult and very hard, so debugging these things have been fairly difficult because you want a tool to actually visualize the graph, and there aren’t a lot of tools that do that right now.

And so there's this reflection tool, reflection agent, and the other thing that we talked about earlier is that retrieval is crucial. And again, this has to be kind of neurosymbolic. It has to be able to do RAG-style embeddings retrieval, but it has to be able to look up functions and symbols inside the code.

This is why I do think I may be extrapolating a bit more even if we get into the world of found models that have really, really large context windows. I mean, Gemini always in the millions of tokens. You will still need very specialized things that do lookups like this, because applied to different contexts knowing the functions and treating it more like how it compiles at the end like a graph.

Large context windows, you can totally shoot yourself in the foot with them.

Gary: Yes, because it's easy for the model to... It's actually, you know, the model will bias a lot more towards whatever is at the end, H kind of like a human.

Amjad: Yes, exactly, and so you still need to do context management, and you need to figure out what to put on how to rank memories.

So this agent, every time it does a step, it goes into a memory bank. And then every time we go into the next step, we need to be able to pick the right memories and figure out how to put them in context. If you pick the wrong memories, for example, if you pick a memory that had a bug, or there was an error in it, whatever it might still think that there's a bug.

But if you already recovered from that, you want to make sure that memory of having created a bug is either kind of augmented by another memory of fixing it or entirely removed from the context. And so memory management is crucial here.

You don't want to put the entire memory in context. You want to be able to pick the right memories for the right tasks.

I feel like this is a really concrete rebuttal to situational awareness and that whole sci-fi, you know, AGI is going to kill us tomorrow kind of argument simply because that all is predicated on larger contact window, more parameters, throw GPUs at it and it's going to work.

Like you can't just scale it up. Like you're not going to get what you want from just scaling it up. There is actually a lot of utility in having these agents work one with one another.

Being actually smart about what is the intermediate representation and being able to pull back, you know, sort of model what a human would do. I mean this is sort of like the case study in like, oh yeah, you can't just, you know, scale up everything by 50x and have it work the way that they think it will.

Amjad: Yeah, in many ways, like building a system like that sort of humbles you, you know, sets your expectation about AI and the progress in AI in sort of a different way because yeah, the systems are very fragile. They're really still not great at following instructions.

People talk a lot about the hallucination problem. I think the bigger problem is like just following orders. It's so hard to get them to actually do the right thing.

Gary: What do you think is the path to AGI?

Amjad: So my view in AGI is that maybe we'll get to something called functional AGI, which is we automate all those economically useful tasks. I think that's fairly within reach. I think it's almost like a brute force problem. It's sort of the bitter lesson.

Gary: Right. Do you think it involves doing a lot of work like what you guys did? Like basically building like carefully fine-tuning orchestrations of groups of agents for each task, so doing what you did for programming but doing it for customer support and for sales, for every accounting, every function?

Amjad: Yeah, I think so. And maybe you can eventually put it all into one model. The history of machine learning has been we create these systems, we grow these systems around these models and eventually the model will eat those systems.

So hopefully like everything that we did at some someday there’s like an end-to-end machine learning system that could do it. Tesla, you know, famously you know, had all these logic and whatever and now like, you know, I think after v13, it's just end-to-end training.

And so, you know, eventually we'll get there, but I wouldn't consider it true AGI because you throw something out of distribution at it and it wouldn't be able to handle it.

I think true AGI would require efficient learning, being able to be thrown in an environment with no information at all, being able to understand the environment by examining it, and learning a skill required to navigate that environment.

And LMs are not that; maybe they're a component of that, but they're not efficient learners at all.

You actually demonstrated this because the way you describe LMs are intuition machines, and in order to get them to work in programming tasks, you have to add this layer with symbolic representation like in programming.

And there's a lot of concepts in programming and how computation works, like Turing-complete and all that, right?

Amjad: Yes, exactly. Those are like very explicit classical computer science, classical AI, yeah, we do backtracking and all that. Yes, that's not generalized, that's specialized. I mean incredibly useful, specialized, yes.

So it's only been live for four days, but already people have done a bunch of like really interesting and impressive stuff with it. Do you want to talk about some of the things that you've seen people do with it that are most like surprising and interesting?

Amjad: Yeah, one of my favorite things that I saw was someone who had an idea for 15 years but didn’t have the tools to build it and was able to build it in 15 minutes. He recorded his reaction, and it's like a personal app. He built an app where he can put memories on a map and attach files and audio files to it, memories about his life.

I went to school here and like add a picture, whatever. When the app showed up and he tested it, he was like, he was so surprised. I almost shed a tear on that. I was like, you being able to unlock people's creativity is so rewarding.

And then I want an integration with Apple Photos, or to use it to actually build an export tool.

Jared: Yes, and another user, Meck, built sort of a Stripe coupon tool. So he has a course he runs on Stripe, and he wants to be able to send people coupons, and so he built it in like, you know, 5, 10 minutes.

Actually, I don't think you would be able to build something like that in no-code. You would struggle really hard. You would probably use two or three no-code tools. People use like Bubble on the front end and Zapier in the back end and what have you.

Sometimes I'm surprised the no-code people are actually quite smart and quite hardworking because they figure out how to create these systems using no-code, but it's just actually a lot easier to just generate the code for it.

It's a coding tool for the no-coders.

Amjad: Yes, yes. And so yeah, we're seeing a lot of traction there, which is actually a challenge I think the no-code tools have in general, straddling this line between they start very much no-code and then they find that people keep pushing the limits of what they want to build in these tools.

And then the frustrating part with no-code tools is that if you hit the limits, you're just stuck. Like you can't solve it. And the cool thing is if you were saying earlier, if you can get the no-code people to switch to Repet, maybe initially they don't program at all, all they know how to do is like prompt it.

But then at some point, they're going to like look at the code and they'll realize that they can just edit it and like it isn't that hard, and that's how they like gradually become programmers.

Gary: Yeah, that's interesting. I played around with it to build just like a simple recruiting CRM, which is actually the kind of thing you would have used Airtable for. And one of the suggested when it told me the plans—one of the, oh would you like this feature—was exactly that.

It just like role-based permissions and auth. It's like, oh, that's pretty like a sophisticated prompt or like suggestion off the bat.

That's a $10,000 a month enterprise feature right there that you could just prompt and have it work. It's crazy. I mean, this is like the definition of low bar, high ceiling; like all of the biggest software companies in the world sort of capture that idea really powerfully.

So my favorite thing is these order multiple order magnitude sort of time difference of building something. Someone said they spent 18 months building a startup. They were able to generate the same app in 10 minutes using Repet.

Someone said they spent a year building a certain app that they were able to build it in an hour with Repet Agent. But yeah, I think it will save millions of dollars of human hours. What a time to be alive.

Gary: Guys, can I take Repet Agent and apply it to my existing coding stack yet?

Amjad: Not yet, got it. So again, it's sort of super early. We built the retrieval system that we built to be able to do this. We should be able to throw it into any codebase, index the codebase really quickly, and be able to give it intelligence about the codebase.

The system also has summaries of files and summaries of projects, so we use LLMs to kind of, as we're indexing the system to create these small summaries for the agent to understand what a project is. So we have the infrastructure for it, but that's the next step.

And we also want to add more autonomy for people who want it. So for the team version of this, we want to be able to send it to the background, so be able to give it a prompt and then it forking the project, going and working as autonomously as it can, and then when it's done it sends you a pull request back or if it runs into a problem it comes back to you with a problem.

The other thing I want to do is, you know, the vision for this has been, you know, we have this bounties program. And bounties are where people submit things they want to build or problems they have, and people in our community, users, help them fix it for a certain price.

And I was thinking, you know, agents are not perfect, and so perhaps the agent can also summon a human. So another tool that it has is being able to summon like a bounty hunter.

And so we'll go to the market and ask the creator working with it, hey, I'm running into a problem. Do you want to put some money on it? And we can go, like, you know, grab an expert.

Gary: That’s so clever!

Amjad: Yeah, so it's like, yeah, cool, yeah, put $50 on it, and we'll go to this market. Hopefully, real-time market will say for $50 we have this problem; can you come in?

A human expert comes in as another multiplayer into the system, either helps you by prompting the agent or by going and editing the code themselves.

Gary: I mean, this whole thing of getting the human to be another agent in this greater intelligence orchestration system you have, yes, I'm a big fan of Licklider's sort of human-machine symbiosis, right? That's always been the thing.

You know, I like to talk about AGI and all of that, but I just feel like, you know, computers are fundamentally better by being extensions of us and by joining with us as opposed to, you know, being this, you know, competitor.

Amjad: 100% team human! We need to print t-shirts!

Gary: You had, I guess, sort of mini-Chessy moments earlier this year when we were all blown away by this demo and sort of, you know, you've been working hard on sort of remaking the way all software is deployed and written for some time.

I mean, what did it take to, you know, get to this moment? You did have to do a layoff and reset your org. You know, what happened?

Amjad: Yeah, so last year, we raised a big round. We felt we were making fast progress, and there was a lot of energy, and I felt like I needed to, okay, grow the company.

You know, for a long time, Jared knows for a long time Repet was like tiny. It was actually run out of your apartment for how many years?

Jared: For many years. For like three or four years, and we were like four or five people for like many years.

Amjad: So we started growing in 20 even when you had a lot of users.

Jared: Yes, like you were four or five employees when you had like millions of users.

Amjad: Yes, that's right. So we were always kind of lean, but I thought last year, okay, we have really big ambitions, we got to go hire people. I got to hire executives, I got to create like a management structure, I got to like grow up is what investors were telling you.

Gary: Oh, you got to hire!

Amjad: No, actually I was on my own, but it definitely was the prevalent advice. I mean, you were absorbing this advice from sort of like the world that was that ordinarily advises startups to do exactly that.

That's right, that's right. And it just got really miserable. We had like, you know, multiple layers. We had different meetings where I'm trying to like run the company from. We had like an executive meeting, staff meeting, whatever we had roadmaps, we had planning sessions, and I just couldn't shake the feeling that it was all layering.

It was not work, it was layering. And but right now we don't have a roadmap right now; literally, we work on like three or four things. I'm involved in all of them; I know what's going on there, I know what people are working on, and I think we got a lot more productive by getting smaller, by, you know, flattening the organization.

I think one thing, that's a story that I think we've heard from many founders, and one thing I'm curious to see how this plays out, is I feel like what actually sparked off a lot of manager mode was feeling that people had more ideas to run with and they had like resources to execute on, and you realize that bureaucracy creeps in, and you actually just can't get ideas done as quickly as you want.

Now I feel like everyone's getting rid of middle management, and I'm curious to see if the same thing, the same temptation, I think will happen again. I think we thought it a little bit personally, even, is when you make it easy to go from like zero to one, it actually helps you create more good ideas because you're like, oh yeah, it's actually like I can just get things off the ground really, really quickly.

And so then it'll be interesting to see how people stay. Now you have like the smaller flatter structure, you'll get more ideas for things you want to do, and then staying like disciplined to not go back into, oh yeah, like we should actually do like the 10 things we could possibly be doing versus like the five or six you can keep in your head.

I think is actually a challenge.

Jared: I guess that there's a Waring idea here because there's Parker Conrad's compound startup. The interesting thing about the compound startup is I think they're trying to explicitly make the other product lines feel like a startup and govern like a startup unto itself, which is like sort of the opposite of having like divisional responsibility.

I also think with Rippling and Par, like Parker is known for having this hiring tactic where he only tries to hire a lot of former founders and then puts them in charge of a product line.

Gary: Which has obviously worked really well for Rippling.

Jared: I think it's hard for most people to pull that off because you can't hire like the quality of former founder unless you have, I think unless the company's already like proven successful, or you're just like a top-tier recruiter like Parker is, he's pretty like you know, top 0.1% of ability to recruit really great people.

Amjad: But Parker surely found remoting though, because we he gave a talk at YC Growth when we did this a couple of years ago, and he was still doing support tickets.

Jared: Oh yeah, still is! He told us that Harge hosted him a couple of months ago, actually right over there, and he said that he said basically he loves answering customer support tickets and he will never let it go because it’s his direct line of information to know what’s really going on with the customer.

Gary: Yeah, I mean, that’s the final remoting.

Amjad: I think maybe he's doing the compound startup; he's giving them a lot of autonomy, but he's in the details.

So how did this play out for this AI agent? Like we talked about how you built it technically, how did you build it organizationally? This was a whole big bat. It was totally new technology that like the earlier team wasn't used to working on. How did you pull it off organization?

Yeah, great question. We tried building agents multiple times in the past, and just the technology wasn't there. And finally, when we felt it was there, actually one of our employees, Zen Lee, who started this new incarnation of this, made a demo, and he showed me the demo and it was so simple.

It was just like the agent like calling a couple tools and doing things in IDE, but I could see that it’s finally almost here. I could taste it almost. And in that feeling, just was like, okay, we're going to make this big bad.

And so it created something called the agent task force. So in the task force, it's like people from a lot of different teams. So you have the IDE team present in the task force, you have the DevX team that works on package management and things like that, you have a UX and design component, and you have the AI team.

So you have the AI team at the center, so it's almost similar to a CARA diagram. So we organize it in the same way that the diagram works. The kernel OS is the sort of AI team, and then they're connecting out to all these tools that are created by the tool teams.

And then on top of all of that, you have the product and instead of the UX team that is working on the entry points and how do you structure this, which was very tough as well. The design was tough, and we had like two meetings every week.

On Monday, we had this war room meeting, where Michael, our head of AI, will do like a run, and we'll see what's broken, what's wrong with it. They’ll come up with the priorities for this week.

Then on Friday, we have the agent salon where I do a run and I look at what's working, what's broken. I ask them about their priorities. We might reprioritize something, things I might change some things in the product; we make big changes like rapidly.

And so every week we made a ton of progress.

Gary: What does doing a run mean?

Amjad: Doing an agent run literally actually going through and using the product and seeing where it broke, seeing where it breaks, and figuring out what the priorities in order to fix it, where it broke.

Gary: Brilliant! Did each of the team basically build their own agent as well?

Amjad: Some of them did because some of them you had to. The screenshot tool was an asset because you had to kind of have an AI look at the screenshot, come up with the thoughts, and then return them to the main manager agent.

So the IDE team wrote the screenshot agent, and then the package management team kind of built probably the tech stack set-up type of configuration, which is really cool.

It worked out; the org structure worked out really well. I mean surprisingly well because I think it is similar to how we worked, and at the center was the user, and now the user is the AI.

Gary: What's coming next with the agent? What do you want to add to it? What do you think are going to be the big next leap forwards for it?

Amjad: Reliability, I think the most important thing right now is reliability. Making sure it's not spinning, making sure it's not breaking, and then expanding it to support any stack you would want.

So right now we don't really listen to the user when they give us a stack; we push back, the agent pushes back. It's like, ah, I’m just going to do it in Python or whatever.

But if you really want OTE, so we want to be able to accept user requirements with the stack—should have the poor "GRY" mode WR only write it in list. You know, this modes thing is a really like an April Fool's thing, like poly-ram over-engineered, bad UI, doesn’t care about UI, everything's literally correct but very confusing.

Gary: How about just the interaction? I mean you mentioned Licklider and the whole human-computer symbiosis theory, like is text, like, as far as it goes? Are there other ways that people you think will want to interact with their AI agent?

Amjad: You should be able to like draw in the UI and communicate with the AI by drawing, right? You should be able to say, hey, like this button's not working, maybe move this here, or this file, you know, bro is not, you know, refactor this file, whatever.

So, you know, if the whole thing is a canvas that you can draw on, you can communicate a lot more expressively with the agent. And of course, you're talking, you know, as opposed to typing.

Being able to talk and draw, it's imagine it on the iPad too. We have an iPad app. It could get really, really fun and creative, kind of like a full UI mockup that you would do in Figma. You could kind of hand sketch it and get it to do it like how running a real engineering product team would feel like.

That's right, and then we're going to add like more simpler agentic tools. So right now, the agent kind of is, you know, takes over, and it's like writing everything, but a lot of people just want more agency—more advanced users.

So we want to be able to do like single step or single action agents. So say like, I want to add this feature, show me what you're going to do. I'll do a dry run, show you all the diffs, show you all the packages going to install, and then you'll be able to accept it or reject it.

And that way, your more advanced users will have more control over the code they're writing.

Gary: Om, thank you so much for coming and showing us the future in such a profound way. If I wanted to do this all myself, what would I do?

Amjad: Well, first of all, I want to say it's again barely beta software. If you're brave and you want to test it and give us feedback, go to Repet, sign up for our core plan because this thing is expensive, we can't give it away for free.

And you'll be able to see that module on the homepage that says, what do you want to build today? And then you can go through that and start working with the agents. Just have an idea in your mind; just write a couple sentences. Don't make it too complicated or too technical and get started.

You'll get a feel of how to work with the agent pretty quickly. It should be pretty intuitive. And share with us what you're building—happy to kind of reshare, retweet, whatever people are building with the agent.

Gary: Amazing! Well, it's time to feel the AGI. We'll see you guys next week. [Music]

More Articles

View All
The Art of Meditation (animated video)
What is meditation and how can we use meditation to alleviate our suffering and achieve peace of mind? It’s probably a bit more complicated than most people think. When we meditate, we watch our thoughts while focusing our attention on a certain anchor t…
The Power Of Walking Away
Somehow, many people feel obligated to give away their time and energy to others. But why? Perhaps they feel the need to prove themselves or have the intense desire to be liked? The problem is that by caring too much about opinions of other people, you be…
New Hampshire Summer Learning Series Session 4: Data Informed Instruction
And all right everybody, welcome back or welcome, and hello! My name is Danielle Sullivan, and Barbara Campbell is my co-host today. We are going to be presenting to you on how to enhance teaching with data-informed planning with Khan Academy. Oh, there …
"America's Best Idea" - President Obama on National Parks | National Geographic
Two of your predecessors felt very much the same thing, didn’t they? Teddy Roosevelt walked these very trails through these redwood trees along with John Muir, the father of the American conservation movement, and these granite mountains. They lit a fire …
15 Powerful Books That Make You Smarter
Picture this. Okay, you come home from work, you make yourself a delicious dinner, and then you’re left with an entire evening that you can spend as you like. But how you spend that evening can either leave you with some hours wasted on small unimportant …
Experience the Tomb of Christ Like Never Before | National Geographic
[Music] You are about to embark on a virtual journey to one of the most sacred places on earth. Nestled in the heart of Jerusalem’s Old City, the Church of the Holy Sepulcher is, for the faithful, a place of pilgrimage and worship. For other visitors, it …