> with new techniques and materials on top of old work done in a way that was usual at the time.
But is there any long-lived project for which this isn't true? Linux and the BSDs surely have many components that fall into this category.
> For example there's BString and BList.
BString is a much nicer string class to work with (IMO) than std::string. It lacks some modern conveniences, and it has some unfortunate footguns where some APIs return bytes and some return UTF-8 characters (the former should probably all be considered deprecated, indeed that's a BeOS holdover), but I don't think there's any intent to drop it.
BList could be better as well, but it's still a nicer API in many ways than std::vector. Our other homegrown template classes also are nicer or have particular semantics we want that the STL classes don't, so I don't think we'd ever drop them.
> Haiku also has seams of BSD code where there'd be a project to do Whatever (WiFi, TLS, drivers, etc.) "properly" in a way unique to Haiku
What would be the point of implementing WiFi drivers from scratch "uniquely" for Haiku? Even FreeBSD has started just copying drivers from Linux, so that may be in our future as well. I don't know that anyone ever really considered writing a whole 802.11 stack for Haiku; there was some work on a "native" driver or two at one point, but it was for hardware that we didn't have support for from the BSDs, and it still used the BSD 802.11 stack. Writing our own drivers there just seems like a waste of time; we might as well contribute to the BSD ones instead.
> But is there any long-lived project for which this isn't true?
I don't think any other project like this exists. You're coming up on your 25th anniversary without shipping the release software !
I see that BString itself also uses this weird phrase "UTF-8 character". That's not a thing, and rather than just being technically wrong it's so weird I can't tell what the people who made it thought they meant or what the practical consequences might be.
I mean, it can't be worse than std::string in one sense because hey at least it picked... something. But if I can't figure out what that is maybe it's not better.
UTF-8 has code units, but they're one byte, so distinguishing them from bytes means either you're being weird about what a "byte" is or more likely you don't mean code units.
Unicode has characters, but well lets quote their glossary: "(1) The smallest component of written language that has semantic value; refers to the abstract meaning and/or shape, rather than a specific shape (see also glyph), though in code tables some form of visual representation is essential for the reader’s understanding. (2) Synonym for abstract character. (3) The basic unit of encoding for the Unicode character encoding. (4) The English name for the ideographic written elements of Chinese origin. [See ideograph (2).]"
So given BString is software it's probably working in terms of something concrete. My best guesses (plural, like I said, I'm not sure and I'm not even sure the author realised they needed to decide):
1. UTF16 code units. This is the natural evolution of software intended for UCS-2 in a world where that's not a thing, our world.
2. Unicode code points. If you were stubbornly determined to keep doing the same thing despite the fact UCS2 didn't happen, you might get here, which is tragic
3. Unicode scalar values. Arguably useful, although in an intensely abstract way, the closest thing a bare metal language might attempt as a "character"
4. Graphemes. Humans think these are a reasonable way to cut up written language, which is a shame because machines can't necessarily figure out what is or is not a grapheme. But maybe the software tries to do this? There have been better and worse attempts.
I don't love std::vector but I can't see anything to recommend BList at all, it's all using type erased pointers, it doesn't have the correct reservation API, it provides its own weird sorting - which doesn't even say whether it's a stable sort,
Edit: oww, the *Chars* method family? Well, that one is bad. STL is sort of lucky here as it tried to figure out unicode when it was already well known.
The RISC-V port was done almost entirely by one developer who took an interest in it. It wasn't as though the project got together and decided to prioritize RISC-V over ARM; it was just that someone did a port, and then it got (mostly) upstreamed. Nobody has taken an equivalent interest in ARM, in large part because, well, the developers are all running x86 machines as you might expect, so that's what Haiku gets developed on. If someone comes along (or one of the existing developers takes interest) in working on the ARM port more, we will hardly reject the patches!
Haiku's "raison d'être" is to be a fully-fledged desktop operating system for general use. So, if there aren't native applications to fulfill daily tasks, then Linux ones tend to get ported.
Anyway, as for the question "why Haiku, if it's just ported Linux apps?" well, because Haiku is a much more cohesive system than any Linux distro. Even if every app you are running on it is from Linux (besides Tracker and Deskbar), the entire system underneath those applications remains tightly integrated.
(Haiku developer here.) There is one hardware accelerated driver, for Radeon Southern Islands; but it's third-party, out-of-tree, and not particularly simple to set up and get running. So, not really.
Thanks for the response. Is there much hope for this to change at some point? I would love to eventually use Haiku on actual hardware, and the lack of video acceleration is the main thing that discourages me from giving it a go.
(Haiku developer here.) This is a pretty common misconception, but it isn't true; Haiku doesn't have a "POSIX compatibility layer", it's just natively POSIX under the hood. You can find some elaboration on an old forum thread: https://discuss.haiku-os.org/t/is-haiku-a-unix-like-os/8801/...
I think what I meant is that the OS model of what a program is constrains what we think can be done on it. I can't think of anything that couldn't be done in Haiku that can be done on Windows or a Unix.
When I think of non-Unix and non-MVS i'm thinking more on the line of the IBM i, PalmOS, or the Newton OS. All three are quite alien under the hood to anyone who grew up on a Windows/Unix world.
If that is what he meant than iOS and Android are that, their touch interface allows you to do things which can not be done in the Windows/Unix world which is why the Windows/Unix world is bringing in touch.
Does Haiku/BeOS have a lineage, were they built off of some other OS in anyway or just their own thing which took what it liked from what was already around and started from the ground up?
At the time, Be often talked about it being clean, legacy-free, ground-up etc.
The main _inspiration_ was the Amiga, but not really AmigaOS.
BeOS was built in the still-new C++ but AIUI predates a lot of standardisation of C++ which subsequently happened -- as was Psion's EPOC32 and its later rebranding as Symbian.
Haiku uses the System V ABI (mostly.) So, we're doing the same things Linux and the BSDs are here, simply by using GCC or Clang without any special tuning here.
> I reckon that before trying to claim you've innovated here it might be a good sense check to compare baseline.
The baseline is "what are other operating systems' kernel- and userland-level condition variables APIs?" And none of the ones I looked at had anything like what Haiku has here, they all have something which is the more classical "lock-switched condvars" just like POSIX has.
The API itself does not depend on what memory ordering semantics are any more than a "mutex_lock()" API does. The implementation will be somewhat contingent on it, of course, but those are two separate matters.
> What exactly are the Haiku atomic operations, in terms of the C++ 11 Memory Model?
The atomic_() functions are (on most architectures, x86 included) implemented using GCC/Clang's __atomic_* functions, with various __ATOMIC_* orderings chosen as appropriate. You can see them defined in the system header here: https://github.com/haiku/haiku/blob/master/headers/os/suppor...
> because you're innovating before 2011, you're inventing the model
No, not really? GCC has had atomic builtins since at least 4.1.0 in 2006. The documentation (https://gcc.gnu.org/onlinedocs/gcc-4.1.0/gcc/Atomic-Builtins...) says: "In most cases, these builtins are considered a full barrier. That is, no memory operand will be moved across the operation, either forward or backward." -- which is basically equivalent to today's __ATOMIC_SEQ_CST.
> so Haiku is off in the jungle on its own and everybody else has a map now, figure out where you are on that map first.
We already did that years ago. The atomic_*() functions linked above in SupportDefs.h have been implemented using the C++11-standard GCC builtins since 2014, and the older __sync_* builtins for years before that.
Anyway, the algorithm described in this article, even if Haiku's atomic functions were not 1:1 with C++11-standard definitions (which they are, as noted above), is clearly portable to other OS kernels. So I am not sure what basis your comment has, regardless.
> The atomic_() functions are (on most architectures, x86 included) implemented using GCC/Clang's __atomic_* functions, with various __ATOMIC_* orderings chosen as appropriate.
There is no one-size-fits-all choice here, so "as appropriate" is almost definitionally a mistake. It turns out that "as appropriate" for Haiku means Sequential Consistency except on simple load and store, which have Acquire-release semantics instead for some (unexplained in my brief exploration) reason.
Still that does answer the question of why these structures seem to work for Haiku despite the lack of what you'd ordinarily expect in terms of ordering guarantees, it's eating a Fence for each Entry and a Fence for each mutex operation. It's a steeplechase!
You are correct that deadlocks can be caused by signals occurring before the wait starts, and thus some sort of mechanism to ensure this does not happen is needed, but I explained as much in the article. The point of this API is that the atomic lock-switch is not restricted to just locks; and further, in some situations, no lock-switch is needed at all.
The former is simple enough, and directly equivalent to what FreeBSD's API allows you to do (and what Haiku's API provides as "convenience methods", as the article notes), and is "atomic" -- it just pushes the atomicity up a level and lets the programmer control it more directly:
ConditionVariableEntry entry;
gSomeConditionVariable->Add(&entry);
mutex_unlock(&someLock);
/* (I could unlock more locks here, if needed, I'm not limited to 1) */
entry.Wait();
mutex_lock(&someLock);
The latter case is the more interesting and unique one, and the article references one place it is actually used in practice (team/process creation), though it doesn't give a pseudocode example, so let me try to give one here:
ConditionVariableEntry entry;
someLongRunningOperation.conditionVariable->Add(&entry);
someLongRunningOperation.start();
/* (I can do whatever I want here, no need to Wait immediately) */
entry.Wait();
Since this "long-running operation" is not even started until after the local Entry has been Add'ed to the Variable, there's no possible way for this operation to complete and signal before we have started 'waiting' (because, even if the Wait() call is at the end, it's the Add() call that counts.)
> Since this "long-running operation" is not even started until after the local Entry has been Add'ed to the Variable, there's no possible way for this operation to complete and signal before we have started 'waiting'
What you've got there is a "Happens before" constraint. Your "no possible way" is assuming Sequential Consistency, but I assure you that your CPU does not in fact provide Sequentially Consistent ordering of individual CPU operations across multiple cores.
You need some way to reliably Order the events. It's possible that the vaguely named "atomic" operations in Haiku provide you with adequate ordering, but who knows since they don't specify.
> Your "no possible way" is assuming Sequential Consistency,
I am assuming events cannot finish before they are started, yes! I am pretty sure that even the (in)famous DEC Alpha could not possibly have time-traveling results.
Once again: there is a distinction between the API itself, and the API's implementation. Once the "Add" function returns, the "Entry" is now waiting on the "Variable". How the "Add" function ensures that is entirely abstracted away from the consumers of the API.
The implementation of "Add", at least, is synchronized using a very standard spinlock, just like similar operations are for mutexes, semaphores, etc. not just on Haiku, but on all the BSDs and Linux. We don't even need to think about CPU operations ordering for that, the spinlock takes care of it for us.
I am pretty sure Linux (and every other operating system, ever) would be in all kinds of trouble if you could read stale values from memory protected by spinlocks, so I don't know why you are casting doubt on these things.
> It's possible that the vaguely named "atomic" operations in Haiku provide you with adequate ordering
Hey, you already wrote a comment elsewhere in this thread about this point, and I replied to your comment with a bunch of information proving definitively: they do, in fact, provide that ordering. But in the cases I described in the parent comment here, it does not matter, because here we are talking about API consumption, not implementation.
> I am assuming events cannot finish before they are started, yes! I am pretty sure that even the (in)famous DEC Alpha could not possibly have time-traveling results.
You seem to think this is a joke, but it isn't. Obviously from the CPU's point of view there is no "time travel" but that's cold comfort for users. The Alpha doesn't promise that there's any coherent ordering at all unless you've imposed one, which the sequentially consistent atomics Haiku is apparently asking for do.
To get Sequential Consistency you're paying a very high price on such weakly ordered architectures. On x86 (and x86-64) the relative price is smaller, because everybody is paying for Acquire-release basically all the time, but it's still substantial on larger modern systems.
This price isn't unique to Haiku, but the choice to pay it (almost) everywhere is, at least in terms of operating systems people would be using today.
> The Alpha doesn't promise that there's any coherent ordering at all unless you've imposed one
Yes. But why did you bring this up in this thread about API usage? It's the implementation's problem to make this work out. "Add()" should be the equivalent of a full memory barrier (at least for the condition variable's memory) no matter how that happens internally.
> This price isn't unique to Haiku, but the choice to pay it (almost) everywhere is, at least in terms of operating systems people would be using today.
Haiku is, in many ways, poorly optimized when compared on such details with Linux or FreeBSD, all the developers know this fact, and we make no secret of it. If this was your entire point in the first place, why not just say so?
By the way, as far as I can tell, OpenBSD's kernel atomics (sys/atomic.h) do not have different versions for different memory orderings; in fact they use the older-style GCC builtins and not the C++11-style ones, so they are also using sequential consistency everywhere they use atomics. Is OpenBSD not a "modern operating system people would be using today"?
I guess, ultimately if the argument is we don't care about performance or capability then, you know, fine, although if you don't care it's weird to do two rewrites focused on performance and write a blog post to highlight the work.
AIUI OpenBSD doesn't lean exclusively on those primitives, you might notice for example it has futexes these days. On the other hand I also don't know anybody who runs OpenBSD.
FWIW your response comes across to me as quite rude and patronizing. I assume you didn’t mean this but the fact that you’ve decided to capitalize terms as if you have some sort of true definition for them, plus you dragging this conversation towards the specific complaint you had in another comment (Haiku might use sequential consistency when it doesn’t need to…although it’s not even clear if you’ve done the appropriate work to verify this before popping off). Maybe consider this for next time you comment? If you need an example that demonstrates curiosity rather than smugness look at the first comment in this thread.
The capitalization reflects terms of art in the C++ 11 memory model. So, not my "true definition" but the one provided by the language used, in this case C++. This matters because Haiku is written in languages which use this model (or in some cases languages which don't specify any model but in practice conform to the C++ 11 model)
If you're Linus Torvalds you can insist compiler vendors adjust things as you prefer to some extent, thus the Linux memory model isn't quite the C++ 11 memory model despite the fact that GCC is used to compile Linux and GCC notionally confirms to C11 (and thus has the C++ 11 memory model), much of what Linux does is not conforming to the ISO document. Haiku can't expect the same benefit of the doubt.
The question of whether Haiku needs sequential consistency where it has it is vexed. Hyrum's law applies. The least scary approach might be to follow C++ and provide sequential consistency by default with an opt-out, then introduce use of the opt-out carefully.
I'll respond to your comment anyways but I do want to remind you that I think it's more a reflection of you talking about what you want to talk about rather than being particularly relevant to this thread.
Operating systems are often written in C or C++. Both of these share a formal memory model to provide a set of useful semantics for well-formed programs. For most code the choice to adhere to this model is the right one. In fact, in many cases it can be appropriate to pick more "heavyweight" constructs despite the fact that they can be a bit slower because they are easier to reason about. LLVM's libc used sequential consistency for its shared_ptr implementation for quite a while until it was updated to use a more efficient set of primitives.
On the flip side, sometimes it is not appropriate to use this memory model. Linux has its own because it predates the C(++)11 memory model. Other good reasons to use your own model can be if the standard one doesn't efficiently map to what you are trying to do on the hardware you're on, or if the operations you need to perform are not encoded in the standard. These kinds of things are actually quite common in operating systems, which is why most of them do not strictly conform: C11 has no concept if "this region of code runs with interrupts disabled" or "I need a full serializing barrier for device memory". Haiku chooses to do its own thing, which may or may not be appropriate for its use case. Coming in and claiming immediately that whatever it's doing must be bad is inappropriate and lacks context.
Huh? Happens before subsumes in thread order. Any events that are caused by long running operation must happen after the addition of the event to the condvar.
But is there any long-lived project for which this isn't true? Linux and the BSDs surely have many components that fall into this category.
> For example there's BString and BList.
BString is a much nicer string class to work with (IMO) than std::string. It lacks some modern conveniences, and it has some unfortunate footguns where some APIs return bytes and some return UTF-8 characters (the former should probably all be considered deprecated, indeed that's a BeOS holdover), but I don't think there's any intent to drop it.
BList could be better as well, but it's still a nicer API in many ways than std::vector. Our other homegrown template classes also are nicer or have particular semantics we want that the STL classes don't, so I don't think we'd ever drop them.
> Haiku also has seams of BSD code where there'd be a project to do Whatever (WiFi, TLS, drivers, etc.) "properly" in a way unique to Haiku
What would be the point of implementing WiFi drivers from scratch "uniquely" for Haiku? Even FreeBSD has started just copying drivers from Linux, so that may be in our future as well. I don't know that anyone ever really considered writing a whole 802.11 stack for Haiku; there was some work on a "native" driver or two at one point, but it was for hardware that we didn't have support for from the BSDs, and it still used the BSD 802.11 stack. Writing our own drivers there just seems like a waste of time; we might as well contribute to the BSD ones instead.