>> I'm curious how much of this depends on the driver behaving sanely at load and unload time, cleaning up after itself, gracefully handling unexpected initialization-time state, etc.
> Well, drivers are just processes. So as long as they don't leave the hardware in a totally borked state a re-start of that process should re-initialize the driver and as long as that works I see no reason why the driver would not be able to restart and function.
Yeah. Fair point. I think I was conflating OS architecture with buggy drivers there.
>> It's sad the best option here is Erlang: that results in "yay message-passing" being restricted to large monolithic systems.
> And Erlang is soft real-time as opposed to QNX's hard real-time. For non-interactive backend stuff it doesn't matter that much but for front-end or hardware control it matters a lot.
Ooooh. Good point. Now I think about it, hard real-time and message-passing is... I can see why people like it :) those two things together are something I've been trying to figure out for a very long time.
>> is your code still in use out there, and if so, what sorts of things would I be doing (banking, using electricity, etc etc) if I were indirectly using it?
> I never released other than some very fancy routers where it was used under the hood without anybody being the wiser, but just about every industrial plant runs QNX somewhere, and quite possibly still the 16 bit version on ancient Compaqs.
Nice. Very nice :)
(I noticed from a comment elsewhere that it was a 32-bit reimplementation when QNX was 16-bit, and that you dropped it in without anybody realizing. That is both insane and awesome.)
>> I guess the next question would be, is an open reimplementation of QNX a sane idea?
> You're asking the guy whose front page post reads 'how to sort 2 tons of lego'? :)
Oooh! That was you! That was a really impressive design. I found the "jet of compressed air" so reminiscent of "use a desk fan to blow the box off the conveyor" in that apocryphal "$2m consultant" story :)
> I'm not always the best in making decisions when it comes to what makes economic sense.
Sometimes economic sense isn't the best driver of things that become revolutionary ideas. Linux isn't overall a bastion of best practices nowadays (sadly) but it was started as "just another hobby project" that happened to go viral (in the 90s, no less).
But even before that, UNIX itself was purely academic and its designers retained the "information wants to be free" mentality, continuing to quietly release updates via tape even after the licensing wars began.
But besides all of that, sitting around all day and figuring things out without commercial focus (at least initially) - basically just having fun being creative - can be the most educational. Oh, and fun. :D (I've only just recently figured this out...)
> But I think the world of computing would be a much better place if we went microkernel. Stability and reliability and real time in my book easily trump performance.
I don't fully understand why Linus thinks that a monolithic kernel approach is honestly more efficient. I can't help but wonder if it was a case of "it was easier for me to do it that way, shhhhhh", or what. QNX is a microkernel that also offers hard real-time guarantees. I find it difficult to believe that microkernels do not offer practical performance in the face of such capability. But then again, I'm admittedly not an OS expert. (Although https://en.wikipedia.org/wiki/Talk%3ANanokernel - thanks Google for showing me the talk page instead of the article! haha - cleared things up nicely.)
I can very easily understand microkernels being more stable and reliable. (Does QNX have anything like Minix's reincarnation server?)
Regarding realtime, I've personally been particularly interested in UI design for as long as I can remember, and one of the things I've been trying to crack for ages is how to get rock. solid. animation. - with consistent fps regardless of what else is going on - and I've finally concluded that there are no alternatives to doing this from the kernel up.
Also - being able to do realtime hardware I/O from hacked-together Perl sounds like... yes please!
>> how set-in-stone are QNX's various APIs - ie, how much of a non-moving target are they, how well-documented is everything, how well-known are the bugs?
> QNX is incredibly simple at the core.
That's promising!
> The hard part was to get the interrupt handlers work right, after that it was a walk in the park. But I do recall having a footswitch connected to the reset button because I got tired of diving under the table to reset the machine. No VMs back then...
Wow, hah. (Hopefully this was on an embedded box with no/a skippable BIOS? :P)
Nowadays a decent i7 (a decent one) can get 200MHz emulated performance in Bochs, AFAIK.
FWIW, MARSSx86 is a fork of QEMU 0.14 with a 200kHz cycle-accurate simulator/instruction logger that kicks in either when you run a command in QEMU's console or in QEMU's launch parameters. So when it's off QEMU runs at full (emulated, non-virtualized) speed. https://github.com/avadhpatel/marss/graphs/contributors shows that development wound down ~halfway through 2012, with very minor updates since. But it's not dead.
Obviously these tools are not perfect, but seem like they might make "serious"/in-depth OS development a tiny bit more fun and accessible. And besides these there's also DOSBox's built-in (and eminiently-hackable) debugger that you can enable at compile time (IIRC).
I also remember stumbling on a little assembler-emulator for DOS that could "play" the assembly backwards, which I've always thought was quite cute. It's easiest to link to the last time I mentioned it (you'll see why): https://news.ycombinator.com/item?id=13807599
>> Obviously binary compatibility itself would be a good idea for an open-source reimplementation of QNX, there are quite a few more apps for QNX than BeOS.
> I would not care about that at all, but I'm sure you could do a ABI layer if you really wanted to.
Oh, okay. Hm.... could this ABI layer itself be a server to the microkernel? :D :D
>> "...dude. No. Too much work. Don't go there." in terms of the investment/reward, with all the details taken into account,
> Fortunately when I started that trick I didn't know just how hard it was. And eventually QNX did come out with a 32 bit version (2 years later or so).
Heh.
This is very very interesting, then. Depending on how long ago you did this (sounds like it was a while ago - not sure when QNX went 32-bit) you might remember quite a bit if you tried to do it again.
And if you did, you've already done the "re-do QNX, fixing all the less-than-ideal bits that couldn't be identified as such at design time" thing - so you'd be going over it a second time, with potentially even more refined results.
> But an open source version of it would make big inroads into all kinds of embedded systems. But good luck getting the likes of NVIDIA to make a driver.
Eheheheh. The situation is not nearly that negative now! :D
First, you target Intel. This is simply tedious: there are real specifications floating around, along with a full GPLv2'd implementation in Linux that "just" needs normalizing (basically, distilling KMS and DRM out of the picture). As you go along, you may be able to get some assistance - Intel recently said hello to FreeBSD: https://news.ycombinator.com/item?id=13848258, and I think you could describe QNX to be sort of on the same spectrum/track as an OS ecosystem in some ways, so that might translate to not-entirely-closed-doors where/when it would count. HD Graphics is a huge win to have available; many embedded systems (:D :D) don't have discrete GPUs, and the on-chip GPU specifies two video heads (IIRC), can handle 1080p+, and generally from the standpoint of "reimplement QNX" provides an overkill level of hardware capacity/capability. It would be more than enough to get in the door of making a serious MVP (that's actually usable), if you will.
With some kind of established thing going on with Intel, work would begin to try and target AMD, specifically the APUs. This is likely where the NDAs would start, but it would be viable to do so because a) AMD would be able to see the Intel-only situation and want to compete and b) I expect there would be some sort of commercial thing with the OS happening at this point (eg, the MVP attracting some bites, either from VCs or industry) helping things along.
From there, it'd be a matter of going "hey NVIDIA...?" and either waiting 2E14 years^W ^W for a port or getting specs to help out.
I realize I've naïvely just described a very high-level timeline involving tens of man-decades of development, a noteworthily large team, and probably a 7 figure budget. Sounds like a ridiculous amount of fun :)
I'll also admit that my experience with C is not quite where I'd like it to be, and I've never written about an OS before so I'd have no idea where to start.
But I'm game if you ever feel like doing this at some point. I was thinking of maybe making a website (hmm, what to call it - I have no idea) so people could have a hub/point of reference to point to and figure out what to work on and stuff.
That being said, no rush; this sounds like a fun idea, best to keep it that way. So TL;DR get in touch if you ever want to pursue this :P
(To clarify - I'm not saying you have to email to reply to this comment, just that my email's in my profile if you want to get in touch in future.)
> Well, drivers are just processes. So as long as they don't leave the hardware in a totally borked state a re-start of that process should re-initialize the driver and as long as that works I see no reason why the driver would not be able to restart and function.
Yeah. Fair point. I think I was conflating OS architecture with buggy drivers there.
>> It's sad the best option here is Erlang: that results in "yay message-passing" being restricted to large monolithic systems.
> And Erlang is soft real-time as opposed to QNX's hard real-time. For non-interactive backend stuff it doesn't matter that much but for front-end or hardware control it matters a lot.
Ooooh. Good point. Now I think about it, hard real-time and message-passing is... I can see why people like it :) those two things together are something I've been trying to figure out for a very long time.
>> is your code still in use out there, and if so, what sorts of things would I be doing (banking, using electricity, etc etc) if I were indirectly using it?
> I never released other than some very fancy routers where it was used under the hood without anybody being the wiser, but just about every industrial plant runs QNX somewhere, and quite possibly still the 16 bit version on ancient Compaqs.
Nice. Very nice :)
(I noticed from a comment elsewhere that it was a 32-bit reimplementation when QNX was 16-bit, and that you dropped it in without anybody realizing. That is both insane and awesome.)
>> I guess the next question would be, is an open reimplementation of QNX a sane idea?
> You're asking the guy whose front page post reads 'how to sort 2 tons of lego'? :)
Oooh! That was you! That was a really impressive design. I found the "jet of compressed air" so reminiscent of "use a desk fan to blow the box off the conveyor" in that apocryphal "$2m consultant" story :)
> I'm not always the best in making decisions when it comes to what makes economic sense.
Sometimes economic sense isn't the best driver of things that become revolutionary ideas. Linux isn't overall a bastion of best practices nowadays (sadly) but it was started as "just another hobby project" that happened to go viral (in the 90s, no less).
But even before that, UNIX itself was purely academic and its designers retained the "information wants to be free" mentality, continuing to quietly release updates via tape even after the licensing wars began.
But besides all of that, sitting around all day and figuring things out without commercial focus (at least initially) - basically just having fun being creative - can be the most educational. Oh, and fun. :D (I've only just recently figured this out...)
> But I think the world of computing would be a much better place if we went microkernel. Stability and reliability and real time in my book easily trump performance.
I don't fully understand why Linus thinks that a monolithic kernel approach is honestly more efficient. I can't help but wonder if it was a case of "it was easier for me to do it that way, shhhhhh", or what. QNX is a microkernel that also offers hard real-time guarantees. I find it difficult to believe that microkernels do not offer practical performance in the face of such capability. But then again, I'm admittedly not an OS expert. (Although https://en.wikipedia.org/wiki/Talk%3ANanokernel - thanks Google for showing me the talk page instead of the article! haha - cleared things up nicely.)
I can very easily understand microkernels being more stable and reliable. (Does QNX have anything like Minix's reincarnation server?)
Regarding realtime, I've personally been particularly interested in UI design for as long as I can remember, and one of the things I've been trying to crack for ages is how to get rock. solid. animation. - with consistent fps regardless of what else is going on - and I've finally concluded that there are no alternatives to doing this from the kernel up.
iOS got fluid animation on a 412MHz CPU, 137MHz LPDDR and 60MHz GPU (http://blakespot.com/ios_device_specifications_grid.html) by doing screen updates with hard realtime, AFAIK. What more needs to be said? :P
Also - being able to do realtime hardware I/O from hacked-together Perl sounds like... yes please!
>> how set-in-stone are QNX's various APIs - ie, how much of a non-moving target are they, how well-documented is everything, how well-known are the bugs?
> QNX is incredibly simple at the core.
That's promising!
> The hard part was to get the interrupt handlers work right, after that it was a walk in the park. But I do recall having a footswitch connected to the reset button because I got tired of diving under the table to reset the machine. No VMs back then...
Wow, hah. (Hopefully this was on an embedded box with no/a skippable BIOS? :P)
Nowadays a decent i7 (a decent one) can get 200MHz emulated performance in Bochs, AFAIK.
And there are tons of other emulation and simulation options out there: on my way to relocating http://www.marss86.org/ I turned up https://github.com/cap-lab/tqsim and http://www.unicorn-engine.org/. And those were just what I unintentionally stumbled on, not what I was deliberately looking for.
FWIW, MARSSx86 is a fork of QEMU 0.14 with a 200kHz cycle-accurate simulator/instruction logger that kicks in either when you run a command in QEMU's console or in QEMU's launch parameters. So when it's off QEMU runs at full (emulated, non-virtualized) speed. https://github.com/avadhpatel/marss/graphs/contributors shows that development wound down ~halfway through 2012, with very minor updates since. But it's not dead.
Obviously these tools are not perfect, but seem like they might make "serious"/in-depth OS development a tiny bit more fun and accessible. And besides these there's also DOSBox's built-in (and eminiently-hackable) debugger that you can enable at compile time (IIRC).
I also remember stumbling on a little assembler-emulator for DOS that could "play" the assembly backwards, which I've always thought was quite cute. It's easiest to link to the last time I mentioned it (you'll see why): https://news.ycombinator.com/item?id=13807599
>> Obviously binary compatibility itself would be a good idea for an open-source reimplementation of QNX, there are quite a few more apps for QNX than BeOS.
> I would not care about that at all, but I'm sure you could do a ABI layer if you really wanted to.
Oh, okay. Hm.... could this ABI layer itself be a server to the microkernel? :D :D
>> "...dude. No. Too much work. Don't go there." in terms of the investment/reward, with all the details taken into account,
> Fortunately when I started that trick I didn't know just how hard it was. And eventually QNX did come out with a 32 bit version (2 years later or so).
Heh.
This is very very interesting, then. Depending on how long ago you did this (sounds like it was a while ago - not sure when QNX went 32-bit) you might remember quite a bit if you tried to do it again.
And if you did, you've already done the "re-do QNX, fixing all the less-than-ideal bits that couldn't be identified as such at design time" thing - so you'd be going over it a second time, with potentially even more refined results.
> But an open source version of it would make big inroads into all kinds of embedded systems. But good luck getting the likes of NVIDIA to make a driver.
Eheheheh. The situation is not nearly that negative now! :D
First, you target Intel. This is simply tedious: there are real specifications floating around, along with a full GPLv2'd implementation in Linux that "just" needs normalizing (basically, distilling KMS and DRM out of the picture). As you go along, you may be able to get some assistance - Intel recently said hello to FreeBSD: https://news.ycombinator.com/item?id=13848258, and I think you could describe QNX to be sort of on the same spectrum/track as an OS ecosystem in some ways, so that might translate to not-entirely-closed-doors where/when it would count. HD Graphics is a huge win to have available; many embedded systems (:D :D) don't have discrete GPUs, and the on-chip GPU specifies two video heads (IIRC), can handle 1080p+, and generally from the standpoint of "reimplement QNX" provides an overkill level of hardware capacity/capability. It would be more than enough to get in the door of making a serious MVP (that's actually usable), if you will.
With some kind of established thing going on with Intel, work would begin to try and target AMD, specifically the APUs. This is likely where the NDAs would start, but it would be viable to do so because a) AMD would be able to see the Intel-only situation and want to compete and b) I expect there would be some sort of commercial thing with the OS happening at this point (eg, the MVP attracting some bites, either from VCs or industry) helping things along.
From there, it'd be a matter of going "hey NVIDIA...?" and either waiting 2E14 years^W ^W for a port or getting specs to help out.
I realize I've naïvely just described a very high-level timeline involving tens of man-decades of development, a noteworthily large team, and probably a 7 figure budget. Sounds like a ridiculous amount of fun :)
I'll also admit that my experience with C is not quite where I'd like it to be, and I've never written about an OS before so I'd have no idea where to start.
But I'm game if you ever feel like doing this at some point. I was thinking of maybe making a website (hmm, what to call it - I have no idea) so people could have a hub/point of reference to point to and figure out what to work on and stuff.
That being said, no rush; this sounds like a fun idea, best to keep it that way. So TL;DR get in touch if you ever want to pursue this :P
(To clarify - I'm not saying you have to email to reply to this comment, just that my email's in my profile if you want to get in touch in future.)