The OS I've Been Working On - Vlog #1

Hey guys, this is mad kids101, and today I'm going to be talking about the operating system that I've been working on for the last two months or so. Uh, but before I talk about what it is and why I'm making it, I'm going to talk about a little bit of the background and what gave me the idea to make this operating system.

So starting maybe at the beginning of the school year, um, seven, five, six, something like that months ago, um, my friend John and I— and John is the other person who makes Mackett's videos sometimes— we started making websites. When you make a website, most of you probably know that when you're writing the code that the user sees, you use languages called HTML, CSS, and JavaScript. Those are the three ones you use, and they all serve a different purpose, and you usually have to use all of them.

Um, but there's a large variety of programming languages you can use for basically the server side, for the stuff that runs on your machine that you have control over that might do some more busy work or manage private information that you don't want the user's browser to have access to. For that, there's options like PHP, you can use C, you can use even Objective-C if you're so inclined, Ruby is a popular one with Ruby on Rails, Python is used. But the technology that I've been using for writing these servers is something called Node.js, which is a technology where you use JavaScript to write the server for your web application, as well as, um, the client-side stuff that the user sees in their browser.

So I still use HTML, CSS, and JavaScript for my entire website. It's just now I have JavaScript running on the user's browser and on my server that I control, that manages, you know, personal information and everything like that. So Node.js is what got me exposed to the idea that JavaScript can be used for all sorts of things; it's not just for web applications that the user interacts with in their browser.

And so I took this idea of JavaScript and I realized that it would actually work pretty well in an operating system environment. So one of the things to note about Node in particular, how it uses JavaScript, uh, to its full potential, is that in Node, when you want to read a file, look up something in a database, you don't say, "Give me the thing in the database" and then wait for it to come back and then process it. You say, "Get me that thing in the database," then you go on and do other things, and then you get notified when you get the thing back from the database.

So that's called asynchronous I/O. It's asynchronous because the two operations that you're doing aren't necessarily synchronized; there's just message passing between two sides or, you know, however many, you know. Um, so Node uses JavaScript to do this because JavaScript is actually, um, a really nice language for getting these callbacks. It's really easy to save the context you're in, so you request a piece of information for the database for a specific user, and when you get the information back, it's really easy to remember, "Oh, I was doing it for this specific user; now I'll send them the data that I just got."

Um, in a language like C or C++, it's not as easy to do that kind of event-driven communication. And so JavaScript was a really elegant choice for Node and for this asynchronous model. Basically, the goal I decided to strive for in an operating system was to make everything asynchronous and, uh, to make everything pretty fast and lightweight. But basically, basically the big key was to split things up into different parts that would talk to each other through events and not by waiting for data to come back.

So basically, two months ago, I decided to sit down and basically refresh myself on how to write an operating system. So this would mean doing a lot of research, writing a lot of code that just doesn't work, experimenting, debugging, figuring out problems, figuring out what I can and can't do. Um, and I wouldn't be using JavaScript for this learning experience, and I haven't been for the last two months. I've been using exclusively pretty much C and assembly, uh, which are what the Linux kernel is written in, and I've been using them to write little components that run on a computer as an operating system that do various things so I can experiment and learn how to write an operating system.

And, uh, it's actually gone pretty well. I've made a system which it doesn't run JavaScript yet, but it has various features that, um, I'll need to run JavaScript and that will make it possible to do everything that an operating system does. And I've, I've done it in about two months; it might be a little less, but I can certainly check my GitHub or something like that. But my project has actually gone pretty well.

So I've called this project Learn OS because it's a learning experience, and I plan to pretty much push it forward until I have JavaScript running in it. Once my operating system—once Learn OS—can run some form of JavaScript and maybe get the JavaScript doing some message passing and have a keyboard driver and all that stuff, I'm going to stop with Learn OS and I'm going to start a new project from scratch, which I'll already know what I'm doing, so it'll be a lot faster, and it'll probably be a lot more well written.

I'm going to start working on the actual JavaScript operating system soon after I finish Learn OS. So now, without further ado, I'm going to show you what Learn OS does so far, and then I'm going to talk about what it does pretty much just a little bit, and, uh, I'm gonna keep making video updates soon, probably, um, within the next two weeks. I'll make another video update on how Learn OS is coming and if I think I'm gonna finish it soon or something like that.

Um, so let me go ahead and show you a demo. So the way I'm going to be showing you this operating system is by running it in what's called a virtual machine, which is essentially just a program that I can run on my regular computer that will emulate physical hardware of a physical computer, so that I can just use my screen recording software and show you what's going to happen.

But first, let me go ahead and set up the virtual machine to emulate a real computer better. So I'm giving it eight processors, 512 megabytes of RAM is reasonable, I'm enabling some features which it uses, and now I'm going to go ahead and boot it up. And by the way, this operating system is actually 64-bit Intel only, so it will only run in 64-bit machines, which made it a lot easier to develop.

And let me drag in the thing. This is something called the bootloader; I didn't write this, but it's just gonna launch my operating system, and here it is. And whoa, I forgot to get rid of that little debugging piece, but you can just ignore that. So, what the operating system does right now is it loads up, it calculates how much memory there is, it discovers what pieces of hardware the system it's running on has, and part of that is discovering how many, uh, CPUs it has.

And it initializes each processor separately because it has to. So here you see the first processor is already initialized, so it does seven processors here, and as you can see I gave it eight processors total, so that makes sense. And once it initializes all the processors, sets up the memory, and does various underlying things which I won't get into too much, it starts running a couple programs. The programs are basically just so this terminal works.

So the first program it runs is something called the message daemon, which is like a registry for services, and I'll explain that in a second. Then it loads something— um, well, I'll just go into the abstract part— it loads a keyboard driver so that it can process keyboard input, and then it loads a terminal program. And these all—these programs find each other through the first process it launched, which is the message daemon, which you can see here: message D now running.

So it launches the message daemon, then the keyboard driver, and then the terminal, and they all find each other. And here it also launches a little program I made to test some threading, which some of you programmers may know, but, uh, that's gone in the final release. This is just a version from a little while ago—this is just a test to see that it works.

But essentially, what we have here is a little terminal where I can type a few select commands that I've implemented just to demonstrate various features of the OS. So, for instance, there is a sleep command which takes a number of microseconds in hexadecimal. So this actually represents about one second, and it delays for a second. And if I sleep for too long, let's say 16 seconds, what I can do is I can press Ctrl+C to terminate the program.

And essentially, the way this works is the terminal forks off into a separate process and starts running it whenever you run a command, and the terminal is still listening for Ctrl+C from the keyboard. So when you press Ctrl+C, it kills the process it just launched, and then it'll show the return status of the process, which I won't get into either that much. But essentially, the terminal still runs, and it's also running the command on top of it.

So that's the sleep command—very basic. There's another command called echo, which just prints out what you type. As you can see, there's a mem usage command which says how much memory is being used. Once again, the number is hexadecimal. Um, and actually, uh, I'm gonna see if I can make this number go up. What happens is it starts to load programs dynamically into memory, so that way, um, it doesn't use more memory than it has to.

So as I start to run more commands, it'll actually— you'll see the memory went up by one page here because it started loading a new command dynamically into memory. So I can see how much memory is used, and the reason I have this mem usage command is if I keep running a command again and again, I would expect not to see the memory go up anything because every time a command runs, all its resources should be deallocated once it stops running.

So this helps me check to make sure no memory is leaking anywhere, and to make sure of this I have an al locker command which at some point allocates, I believe, 32 megabytes, and it zeros it all out and then it terminates itself without freeing it. But despite all this, if I run mem usage, you can see there's one more page used because it had to load this new program and whatever. But if I run it again, the memory did not increase anymore.

So you can see that even though it's allocating a significant amount of memory, uh, none of it is being leaked, and it's actually deallocating everything perfectly. So this is a pretty straightforward system that I have here. There are two other commands to demonstrate something called return values, which is basically the status of a program after it ran. So all these programs that I've been running, like al locker, mem usage, have been running fine.

Um, you saw here when I hit Ctrl+C on the sleep, it did print this out in red because that indicates that the task was terminated. When a task has a problem, for instance, if a task tries to access some piece of memory it doesn't own or tries to, you know, break into your system or whatever, it will fail with an error, and the terminal then gets that error message back and can actually display it.

And here's a different kind of error; there are several different error codes that can possibly happen. So this is basically all the operating system does right now. Additionally, if I type a command that doesn't exist, it says "no command found." But the key features of this operating system are that even though it's not JavaScript, it is completely event-driven. So when I run a command, the terminal app isn't using any CPU until it gets an event that the command finished, whether it finished with an error or not.

So it is event-driven, and these various programs— you see the terminal talks to the keyboard program, and when you run a program on top of the terminal, that program talks to the terminal in order to talk to the operating system to print things out to the screen. So programs are capable of talking to each other, just like I plan to do in my operating system, and they're all— pretty much everything is running as a program; none of it is built into the underlying kernel of the operating system.

But anyway, this is the basically Learn OS and its current state. Um, there's a few more features I'm going to add, and then I think at some point I'll actually be able to get a good JavaScript engine running, so I can start running JavaScript.

So I'm really excited about this. I thought I would make this video to update you guys and show you what I've been working on. So thanks for watching, Mac Heads 101. Subscribe, and goodbye!