There is a spectrum of solutions for dealing with slow I/O in programming languages (well, all I/O is best assumed to be slow I/O). At the two ends of the spectrum are:
- continuation-passing style (CPS), hand-coded or compiler
- preemptive threading / processes
In between lie various solutions, like async/await (closer to CPS), and green threads (closer to preemptive threading).
The key difference between the two ends of this spectrum is memory footprint. With CPS you manually compress state into tuned structures. With threads you store state all over the stack in very inefficient ways because all those stack frames take extra space.
Any solution towards the thread side of the spectrum will yield significantly larger memory footprints than solutions towards the CPS side of the spectrum.
On the other hand, solutions towards the CPS side of the spectrum can require writing application-specific schedulers if fairness issues arise.
On the whole, solutions on the CPS side of the spectrum are better, IMO.
Java is kinda stuck with threads, so green threads make some sense. Of course, you can get pathological issues in M:N threading, so be careful about that.
This is partially right. But the Java implementation under the hood is still a CPS transformation, so it only allocates the memory that is required for the "virtual stack". Compared to that Go - which has similar semantics - allocates a more classical stack for each goroutine, which can however grow over time.
The CPS approach is certainly more space efficient, but but I'm not sure how much of a difference it really makes in the end. Go seems to be doing well too, after some attempts with linked stack segments and then moving to copying stacks while growing them.
What is interesting is that that some of the CPS like implementations have other performance drawbacks. E.g. since it makes the virtual stack more distributed over memory, the cache efficiency of such an approach might be lower. In Rust one limitation of the CPS approach is that the coroutine state is first stack allocated before being moved onto the heap, and this operation has shown itself to be costly for some applications. So right now I'm not sure if there is any implementation which is superior in all possible benchmarks. But the Java one definitely seems to make a lot of sense for what they want to offer!
Normally a POSIX thread gets a very large stack (1MB is typically the default), and obviously these "virtual stacks" will only be as large as they grow (splayed on the heap?). But the memory for those POSIX thread stacks isn't necessarily allocated up front! The OS grows the thread when the thread traps trying to access the guard page, so it's not really a 1MB stack.
Now, if you need to serve 1e6 clients with threads, and those are 1MB stack threads, then you'll be using 1TB of your VM space, which... is almost certainly going to have some performance issues (MMU table size issues at the very least). If you splay your stacks on the heap as linked lists of stack chunks then you might get away with having a very large (and fragmented) heap with large page table entries, which might be a win.
I think approaches on the CPS side of the spectrum will be generally better than this. No, I don't study this and I don't have numbers. Yes, CPS in general means allocating closures on the heap so that some state does live splayed all over rather than compressed, but it doesn't have to be so. But often you'll have only a handful of such closures, and the language could understand that they are one-time use closures (hello Rust) so that no GC is needed.
I've written a small (proprietary) HTTP server that is hand-coded CPS -- specifically it supports hanging-GETs with Range: bytes=0- of regular files as a form of tail -f over HTTP, which is great for log files. That implementation has a single object per GET that has all the state needed, and the only other place state lives is in epoll event registrations (which essentially are the closures, and they are very small, and only one per-connection). Granted, this is a very simple application, and it would be a lot more complicated if, for example, it had to do async I/O directly on a block device to implement a filesystem in the same process -- that would require more care to keep the state compressed.
So in general I'm for CPS. But it's generally true that CPS solutions cost more dev time, and that can be prohibitive. The memory footprint cost difference will be a linear factor, which does not trivially justify the additional dev cost. Then again, if you'll be running lots and lots of instances with lots and lots of clients, the run-time savings can then easily be gargantuan compared to the dev costs -- but no one measures this, and by the time you wish you'd used CPS it will be too late and reimplementation costs prohibitive. Then again, async/await might fit the bill well enough most of the time.
> Any solution towards the thread side of the spectrum will yield significantly larger memory footprints than solutions towards the CPS side of the spectrum.
The community-at-large decided that hand-tuning garbage collection was too finicky and not worth it, even though it obviously 'costs' memory.
I'm frankly at a loss as to why so, so, so many blogposts and tech experts are all-in on the CPS-side of this argument; it seems quite obvious to me that in the vast majority of cases, the considerably simpler* model of (green) threads means you're making the exact same trade-off: Simpler to write and debug code at the cost of needing more memory when running the app you write.
*) For sequential/imperative-style languages, that is. If you're writing in a language that is definitely and clearly intended to be written in a functional style, I can see how the gap between CPS-style and threading-style is far narrower. However, java, python, javascript - these are languages where the significant majority of lines of code written are sequential and imperative in nature.
Also note that in e.g. java you can actually configure stack sizes as you make threads. Thus, your choice of words of "'significantly' more memory footprint" is debatable.
A GC does not obviously cost memory. It might, or it might not. Both a GC and a traditional memory allocator have hidden costs. A GC with movable objects can sometimes do better, because it can manage fragmentation.
I prefer a message-passing style; on which side of the spectrum would this fall?
My experience with green threads is libraries that make I/O operations look like a regular function call. This is similar to RPC where a remote call and a local can look the same, even though the remote call is much slower. This can result in surprising performance characteristics. Even worse, a remote call can time out or take indefinitely long to complete; the same is not true of local calls. Message passing is more onerous but makes surprises more obvious.
I've often fancied writing for an architecture where main memory is treated as fast remote storage, accessible with message passing. I know such architectures exist but I've never had the opportunity to write for one. I wonder if the change in style would have a positive or negative effect on performance.
In CPS the continuations (closures) have to be allocated on the heap, but generally they are one-time use only, which means no GC is needed for them. Hello Rust.
> it seems quite obvious to me that in the vast majority of cases, the considerably simpler* model of (green) threads means you're making the exact same trade-off: Simpler to write and debug code at the cost of needing more memory when running the app you write.
I don't find it's quite that simple.
My experience is that the complexity of CPS tends to scale linearly with use, whereas threads scale exponentially. For small uses threads are easier, but CPS quickly catches up.
CPS forces you to actually declare a dependency tree for your data. Things depend on other things, and that exists in your code. It's very easy for threads to end up a mess, where it's not clear how data is passing through the code, which causes bugs like deadlocks and race conditions.
It's deceptively easy to write code where thread A tries to lock mutexes X and Y, and thread C tries to lock mutexes Y and X, and it deadlocks because neither thread can get both locks.
It would be much harder and more arcane to do that in Javascript or in Python's async. I'm not saying it's impossible, but I don't think I've ever accidentally created a race condition or deadlock in their CPS engines.
TL;DR if your functions are only marked async so you can await something, threading probably is simpler. If you're actually passing promises around, things become much more favorable to CPS.
Java is kinda stuck with threads, so green threads make some sense.
This isn't really what's happening here. Firstly, you can implement CPS on the JVM no problem. Kotlin Coroutines do exactly that. Loom's design is a very, very explicit design choice. Ron Pressler - the lead and designer of Loom - has talked about this extensively in many videos. He has argued persuasively for the way Loom works as not only a good way but the best possible way, one which is not a requirement of Java's previous design choices but rather, is only actually possible due to Java's prior design choices.
It's highly recommended. I'll try to summarize the basic argument.
The ideal, from a developer's perspective, is to have the programming model of threads with the efficiency of hand-coded CPS or state machines. Why: because threads naturally provide useful debugging and profiling information via their stacks, they provide backpressure, because there are tons of libraries that work with them and already use them, and most critically because it avoids the "colored function" problem which splits your ecosystem.
Why do most languages not provide that ideal? Mostly for implementation reasons. It's not due to theoretical disagreements or anything. Providing what Loom does is very difficult and is possible largely only because so much of Java and the Java ecosystem is written in Java itself. One reason native threads are relatively heavy is because the kernel can't assume anything about how the code in a process was compiled or what it will do. The JVM on the other hand is compiling code on the fly, and knows much more about the stack. In particular it knows about the (absence of) interior pointers, it knows it has a garbage collector, it controls the synchronization and mutex mechanisms, it controls debugging and profiling engines.
This allows it to very efficiently move data back and forth between a native thread stack and compressed encodings on a GCd heap. It's also why Loom has some weaknesses around calling into native code. Once you're outside the JVM controlled world it can no longer make these assumptions anymore and must revert to a much more conservative approach (this is "pinning" the "carrier thread"). Note, though, that this situation is not worse than async/await colored functions, which have exactly the same issue.
Maybe? it depends how you use them... any asynchronous operation will require memory, and that memory has to go somewhere. In CPS it goes on the heap... in the threaded style you have a stack you can put it on. This probably will waste some memory, though not as much as you might think. On the other hand, you can also reclaim the stack memory as soon as the operation finishes. The CPS technique creates garbage that accumulates until you do gc cycle, but the whole principle of fast concurrent GC is to let garbage accumulate! I'm really not sure what will end of consuming more memory, particularly if you design your threads to primarily use stack allocation, which is admittedly hard in a language like java.
Green threads by definition cannot use OS stack and must allocate their stack memory on heap. Although this memory can be reused, as it is known from Go to avoid performance bottlenecks at least for Go code is better to allocate the stack as single continues block and copy the stack to a bigger block when thread’s stack reaches the current stack size. But then the whole stack space is pinned to the thread and cannot be reused.
For Java it may still be possible not to allocate the whole stack as a single chunk and instead have smaller chunks like one per few frames. But I really doubt that it can reduce memory pressure compared with CSP in real applications especially given how good GC became in Java.
So in Java we know a few things about the stack that are not true for other languages. We know nothing on the stack is a pointer into a Java stack frame, and nothing on the heap points into a Java stack frame. These facts allow us to mount virtual threads onto carrier threads by copying portions of the stack to and from the heap. This is normally less memory than you’d expect because although you might have a pretty deep stack most of the work will happen in just a few stack frames, so the rest can be left on the heap until the stack unwinds to that point.
The big advantage of this over CSP is that you can take existing blocking code and run it on a virtual thread and get all the advantages, there is no function colouring limiting what you can call (give or take a couple of restrictions related to calling native code).
I like CSP precisely because it requires to color-annotate the code so it is knows what can and what cannot do IO! Surely it decreases flexibility, but makes reasoning about and maintaining the code easier.
Thread stacks are not OS level objects, at least in linux you just malloc or anon-mmap some memory and pass that to clone() or you own green thread implementation.
The question is can unused potion of the stack be used for anything else? With native threads the answer is no and so is with Go green threads. Time will tell if Java can pull off the trick of sharing unused space place, but I am sceptical.
With POSIX threads the stack size defaults to something like 1MB or 2MB depending on the platform, but it's not allocated up front -- the stack grows as needed up to that maximum.
The main difference then between allocating stack chunks on the heap as needed, and stacks grown by the virtual memory subsystem, has to do with virtual memory management matters. If you can use huge pages for your heap, then allocating stack chunks on the heap will be cheaper than traditional stacks.
In CPS the state of the program is captured in the continuation, which is a closure, which is allocated on the heap, and in any ancillary data structures pointed to by it.
However, there's generally only a very small number of such closures -- typically only one -- and they are generally one-time use only. That means they can be freed as soon as they tail-call out. Hello Rust.
I've written hand-coded CPS in C, but I take your point. The issue is that while every function exits via tail-calls, so the stack footprint is small, every continuation is a closure that has to be allocated somewhere, and that somewhere is generally going to have to be the heap.
My biggest gripe with continuations, futures and callbacks is the propensity for stalls; if the programmer makes a mistake it's quite likely that the program will just stall, with very little insight available as to why that happened.
With threads, green or not, you have a much clearer failure model and is easier to debug.
But doesn't that depend entirely on the synchronization mechanisms and how safe they are? For example, contrary to what you suggest I'd say that Go's stateless green threads are hard to debug (though there are good tools) and easily lead to deadlocks. That's because the available synchronization mechanisms finitely buffered channels, non-re-entrant mutexes, and a few atomic operations are hard to use correctly. Higher level constructs like actors or transactional memory are in my opinion less error prone.
The point is that e.g. futures are just some threads with a global synchronization mechanism for obtaining the result. Whatever makes the future stall will also make a low-level thread + your own synchronization stall. Or do you mean some more advanced failure-tolerant threading like in Erlang as compared to less advanced threading primitives like futures?
I definitely prefer CPS, especially in functional languages (where the noise I complain about below often fades away entirely).
On the other hand, CPS usually is a bit noisier from the developer's perspective; either your continuations are callbacks (whence callback hell) or your continuations are, as you say, manually compressed, tuned structures, which requires a fair amount of manual labor.
I believe Rust uses a CPS transform (well, more of a continuation-returning style, no?), but it automatically generates the tuned structures ("futures"). The cognitive overhead isn't all gone, but it definitely helps.
What about async/await? You can basically write linear, imperative code, and don't have to deal with continuations or these "tuned structures" manually.
For me it is the cleanest style of writing concurrent code. And more and more I find I can also replace state machines with it, which makes sense because the compiler generates state machines under the hood usually.
You know, the kind of code where you have to communicate with some outside device and it is easy to do blockingly but devolves to state machine madness if you need to do other things concurrently. For example it would be really nice if I could use async/await in C on a microcontroller to read from a serial port...
2. Poor interaction with debuggers, profilers, and other tools that expect to be working with normal stacks.
Loom solves this because it lets you work with normal threads, but suddenly you can have millions of them in a process without blowing out your memory or other related problems.
> You know, the kind of code where you have to communicate with some outside device and it is easy to do blockingly but devolves to state machine madness if you need to do other things concurrently.
Speaking personally, I've found Lua's coroutines to have the nicest experience for modeling flows like that. The big issue with async/await is the function color problem [0] -- writing async functions is perfectly fine, but mixing them with non-async functions can be extremely frustrating. Especially if you're doing anything with higher-order functions.
I used to struggle with "function color" until I realized that the functions just have a different type. Async functions return a future `Task<Thing>`, while normal function return a plain `Thing`. Of course they are incompatible.
A different way of looking at it is that in asyncs functions you should only do things that have negligible runtime (compared to the response time of your GUI or network service). If your task needs more time, you mark the call site and the called function "async" and the task will suspend somewhere "down in the call stack". (Without looking into it too much, I think something similar actually happens with these virtual threads. They modified IO functions to do cooperative multitasking under the hood?)
As to async functions being contagious, I found it helps to split "imperative" procedures and "pure" functions, and the async color mostly applies to the previous.
The problem with function coloring is that it divides the language for no good reason. Should there really be two names for the same sleep function, just because one is blocking and the other is not? As for a return type, that’s just a leaky abstraction imo (especially for voids, like is a blocking call returning nothing different than an async call returning a Future void?)
As for loom, due to it running all in a runtime, a blocking ‘read’ call for example is not actually a blocking system call (everything uses non-blocking APIs at that level) so the runtime is free to suspend execution at such a blocking site and continue useful work elsewhere until that finishes. So for some “async” functionality you can just fire up a new virtual task with easy to understand blocking calls and that’s it, it will do the right thing automagically, and it will throw exception where it actually make sense, you will be able to debug it line by line, no callback hell, etc.
Loom will also provide something called structured concurrency where you can fire up semantically related threads and easily wait for their finish at one place.
As for pureness, I don’t think it maps that cleanly to async/blocking. What about doing the same function on each pixel of a picture in memory where you subdivide it into n smaller chunks and run it in parallel?
Personally, in JavaScript I like that you can mix and match imperative and asynchronous code using Promise instances. It lets you handle asynchronous control flow in a purely synchronous function.
However in other languages, having functions be of a different 'color' is far more painful. In Python for example, a synchronous function has to setup an event loop manually before it can run an asynchronous function. The call works, but nothing is 'running' without the event loop. Additionally, the asynchronous function may have been written to work with a particular eventloop (e.g. trio vs curio), and thus you have to use that type.
If non-blocking code has a standardized control state like Javascript, I think it's better to be explicit about async vs sync.
I also think of "color" fundamentally as different types; it's just painful to have two kinds of functions that you can't combine. I, personally, really feel that async/await is just adding a second kind of continuation (promises/futures) to a language that already has a perfectly good one (call stacks), and the language ergonomics suffers for it.
The reason I say functional languages don't get bit by this as bad is because functional languages rely far less on the specific notion of a call stack, and it's usually much easier to work with continuations (either via primitives like shift/reset or via syntax like do-notation).
I struggle with this idea of "separate colors"... I see Promise returning functions as a super set of immediately returning functions... that is, any function that can return immediately could also return as a Promise (which resolved immediately), so really, the immediately returning function is just an optimization to apply when it's helpful. I'm curious why a language suffers from this explicit separation of code which "returns immediately" versus code which "returns eventually"?
> With CPS you manually compress state into tuned structures. With threads you store state all over the stack in very inefficient ways because all those stack frames take extra space.
On the other hand, these stack frames can be thought of as a large arena allocator for what would otherwise be lots of smaller objects allocated on the heap.
Really excited to see this! Seeing some of the early comments here I think folks may not realize how awesome this would be in the server space.
After all, a big reason that NodeJS won a lot of popularity on the server is that, for many types of common webserver workloads (i.e. lots of IO, relatively minor CPU usage), NodeJS can actually scale much better than Java with its thread-per-request model.
With these virtual threads, though, you could get the best of all possible worlds - a webserver that scales like NodeJS, but without some of the "CPU starvation" issues you can hit in Node if one executing request doesn't yield, and also without having to worry about "function coloring" like you do in Node with async vs. non-async functions.
Really, really fantastic development, have been waiting to see when this would come out.
> For many types of common webserver workloads (i.e. lots of IO, relatively minor CPU usage), NodeJS can actually scale much better than Java with its thread-per-request model
Linux can handle a ginormous amount of threads quite well, would be interesting to see a deeper investigation to this theory.
The problem with doing it all native is that stack sizes are quite variable, especially in managed languages where modularity and code reuse works better, so it's common to have tons of libraries in a single project. The kernel won't object to lots of threads, but once those threads have been running for a while a lot of stack space will be paged in and used.
Loom solves this by moving stacks to and from the heap, where there's a compacting concurrent GC to clean up the unused space.
Yes. I am not as familiar with the underlying implementation of goroutines, but this description in the linked JEP sounds exactly how I understand goroutines to work:
> The JDK implements virtual threads by storing their state, including the stack, on the Java heap. Virtual threads are scheduled by a scheduler in the Java class libraries, whose worker threads mount virtual threads on their backs when the virtual threads are executing, thus becoming their carriers. When a virtual thread parks -- say, when it blocks on some I/O operation or a java.util.concurrent synchronization construct -- it suspends, and the virtual thread's carrier is free to run any other task. When a virtual thread is unparked -- say, by an I/O operation completing -- it is submitted to the scheduler, which, when available, will mount and resume the virtual thread on some carrier thread, not necessarily the same one it ran on previously. In this way, when a virtual thread performs a blocking operation, instead of parking an OS thread, it is suspended by the JVM and another one scheduled in its place, all without blocking any OS threads (see the Limitations section).
Yes. But, since there are _two_ kinds of threads in Java (os and virtual), you still have to be very careful never to block a virtual thread. In Go/JavaScript/Beam, it doesn't matter because you literally can't block a thread (while idle). This is the kind of thing that's not terribly useful until nearly every library you interact with is using it as well.
Also, there's no new syntax, so you're stuck with all the same thread pool concurrency we've been using for decades.
EDIT: It looks like I'm wrong about this:
> My understanding is that you won't have to worry about blocking a virtual thread, because all IO APIs are being modified to park when executed in the context of a virtual thread.
My understanding is that you won't have to worry about blocking a virtual thread, because all IO APIs are being modified to park when executed in the context of a virtual thread.
That said, you'd still need to worry about unsafe code, like JNA/JNI or other such thing that could still block. And I'm not sure there will be a way to prevent long running CPU task from clogging up the virtual thread executor threads.
> My understanding is that you won't have to worry about blocking a virtual thread, because all IO APIs are being modified to park when executed in the context of a virtual thread.
And, from what I read in the original JEP, the underlying system thread pool (which all virtual threads float between as needed) will be expanded when a virtual thread gets pinned, so you don't have to worry about exhausting your pool. (If you pin too many threads, obviously you'll be consuming more OS resources than you may have expected, but that's a different problem.)
What do you mean by pin here? Do you mean that a blocking IO will block the thread, but it will also add one more thread to the virtual thread executor pool? So blocking won't starve your virtual threads?
That's right. From the linked JEP, under "Scheduler":
> Some blocking APIs temporarily pin the carrier thread, e.g.most file I/O operations. The implementations of these APIs will compensate for the pinning by temporarily expanding parallelism by means of the ForkJoinPool "managed blocker" mechanism. Consequentially, the number of carrier threads may temporarily exceed the number of available processors.
Just to clarify, though, most currently blocking IO operations will not pin the carrier thread, because most IO operations you make from a webserver are network calls (e.g. to another API or the database), and those network APIs have been modified to not pin. From just a bit further up in the JEP:
> The implementation of the networking APIs defined in the java.net and java.nio.channels API packages have been updated to work with virtual threads. An operation that blocks, e.g. establishing a network connection or reading from a socket, will release the underlying carrier thread to do other work.
In Go one can block the native thread via using API that use blocking OS calls, like Linux file IO. In this case Go runtime allocates more native threads to run other language threads.
That's case for virtual threads also. It uses ForkJoinPool.ManagedBlocker to add additional threads.
"File I/O is problematic. Internally, the JDK uses buffered I/O for files, which always reports available bytes even when a read will block. On Linux, we plan to use io_uring for asynchronous file I/O, and in the meantime we’re using the ForkJoinPool.ManagedBlocker mechanism to smooth over blocking file I/O operations by adding more OS threads to the worker pool when a worker is blocked."
I learned an hard lesson in the Borland ecosystem.
Always go with the platforms languages, and the IDEs from the platform owners, even if others are more shinny.
Long term it always pays off to be the turtle, as the platforms move into directions not forseen by the shinny objects, and 3rd party IDEs keep playing catching up with SDK features.
Eclipse and NetBeans do exist, and... ehhhhh. I used NetBeans for a long time; couldn't stand Eclipse; and these days I only use IntelliJ. But the others absolutely exist, and it'd be hard to say that Apache and the Eclipse Foundation aren't deeply embedded in the Java ecosystem.
Eclipse is fine. Especially from VsCode where it uses the Eclipse language server. It boots fast, and when you run it with a modern JVM and GC the memory usage is leagues lower than IntelliJ.
If you need multi-platform then coroutines is still your best bet. But many people don't use Kotlin in a multi-platform way, and lightweight threads will be an easier migration path (and more compatible with Java libraries if you cant avoid one) compared to coroutines.
Node.js doesn't create a thread per request; it's single-threaded with evented I/O. You can use node-cluster to start more than a single thread to saturate multi-core CPUs and load-balance HTTP requests across these, but that doesn't make it thread-per-request.
I think my high school English teacher would agree with you that the sentence is written awkwardly (I can see the 'awk' note, in red, on my paper right now :) ). Here's how I parsed it:
> a big reason that NodeJS won a lot of popularity on the server is that, for many types of common webserver workloads [...], NodeJS can actually scale much better than Java with ~~it's~~ [Java's] thread-per-request model.
Twisol was right - I was trying to imply strikethrough using the Markdown syntax in an attempt to depict the idea of replacing "its" with "Java's". It didn't work as well as I hoped. In my mind I can see more 'Awk' scribbles on my post, and looking at it I agree :)
Adding in the 's is 100% my mistake. I've been guilty of using "it's" as the possessive form for most of my life, but that changes today! :)
(Forgive my grammar nazism.) The possessive form of "it" is "its": "The dog wagged its tail". But for basically everything other than pronouns and plurals, the possessive form involves adding "apostrophe s". In recent years, many people have tried to apply this rule to "it". But the problem is that "it's" is understood to be a contraction of "it is" or "it has"; furthermore, "its" already exists as the standard possessive form.
One thing I say to people using "it's" is that by analogy, you also need to say: "He got he's skills. She missed she's ride. They have they's meeting."
Thank you a ton for posting this! I've been doing this for most/all of my life and it didn't really make sense till now. I've had people explain it before but it didn't really make sense. Here's what I got from what you wrote (please correct me if this is wrong / kinda off in some way)
For most words, the possessive form is "<word>'s"
For pronouns (including it) there are different rules. He becomes his, she goes to hers, it goes to its.
Also, words that already end in s don't get the " 's " treatment.
(Question - for words that end in "s", we put the apostrophe after the existing, ending 's', yes?)
Thanks again for posting this - viewing the possessive form of it as (yet another English language) exception to the normal rule of " 's " is really helpful.
> One thing I say to people using "it's" is that by analogy, you also need to say: "He got he's skills. She missed she's ride. They have they's meeting."
This is a great distillation of the intuition I've always had, but never quite verbalized.
Sorry, yes, my sentence was poorly written with the ambiguous antecedent. Most Java webservers use a thread-per-request model, which is why Node can usually scale to more concurrent requests.
I can't think of any framework that still does one thread per request. Normally there is a a queue of incoming requests and they then get dispatched on a thread pool as threads return to the pool.
The challenge is normally that if any of the threads in the pool, as part of processing a request, needs to itself make an IO call, it will block. Ideally you'd want to park the request processing, return the thread to the pool, pick up the next request, until the IO is done where then on the next thread available from the pool you'd resume that request instead of picking another one. This is what the virtual threads will make really easy I think.
Maybe not _all_ possible worlds. You still have original Threads for things that need an actual OS thread. Its not a solution for UI threading.
There will be code that needs a native thread or non-preemptive threading and shouldn't be run on a virtual thread. In that sense there is method coloring but its yet to be seen how common a problem that will be.
Library writers and frameworks will need to sort out patterns for how to call Runnables in a safe way.
Yes but you always need original/kernel threads, regardless of what approach to async you need. The concept of a thread and a stack is hard-wired into the CPU.
W.R.T. code that needs a native thread: at the moment there's only two types of such code. One is code that uses Java's synchronized statement. That's supposedly just a, ehm, small matter of programming to fix. The other is calling into non-JVM controlled code. That's fundamental and no approach to scalable concurrency can fix it, not CPS/async/await or anything else because it's a foreign compiler.
But fortunately the JVM has some really interesting tricks up its sleeve there. For instance you can compile your native code using LLVM and then execute the bitcode on the JVM. Well, OK, currently GraalVM doesn't support Loom but hopefully Graal will be upgraded to do so as Loom gets integrated into HotSpot. And when it does, you will be able to call into code written in C/C++/Objective-C/Rust as long as that code can be recompiled with your own toolchain and as long as you can tolerate it being JITCd, also whilst benefiting from Loom's scalability.
Why do you say CPS can't fix it? C# works around this by having a synchronization context and ways to bounce around contexts. In this way C# async/await is able to ensure code is run on a specific native thread. Is that not a fix?
The idea is you need to understand your workload and run tasks on schedulers meant for that workload. You'd make sure to move that work to a context for long running tasks.
Not unlike how you might use a non-virtual thread pool in Java.... but it seems wrong to imply that you no longer need to think about this stuff.
That’s not function coloring, it is up to the caller whether to start it in a virt thread or a real one. Function coloring is having two methods do the same thing differing only in name and signature (eg. there is a blocking sleep and a non-blocking one).
>it is up to the caller whether to start it in a virt thread or a real one.
Sorta kinda but not when you're working in a framework that will call your code or working in some library where the abstracted code is non-obvious or uneasy to configure.
Maybe its not function coloring, although I wouldn't know what else to call it and I think its quite similar. What would you call the problem?
You should try Quarkus. It is a production framework built by Redhat. It uses Java-GraalVM under the cover to compile your entire webapp to an executable (like golang does).
It's just as fast.
Java is the highest performance and most tuned VM there is. I think you're really thinking of java from a long time ago, if ur thinking this
If you use gargantuan Java frameworks, you'll use a lot of RAM. Just don't do that. With Spring Boot and similar frameworks, the RAM usage is really just very modest. I'll give you startup times, since I am not a believer in Quarkus and Graal. And I wouldn't use Java for a serverless function that needs to spin up and respond quickly. But for a typical (blue/green-deployed) application in my world, startup time is still only a few seconds, which is fine for many applications. And I am not settling for "fine", just saying that the startup time isn't a big consideration, against a lot of things the Java (or Spring, in my case) ecosystem offers me.
You are probably right. It's one of those things where I've not seen a need to jump aboard. I'm still being fearful of reflection going to break on me. Probably irrational fear, but fed by me not understanding how it wouldn't break. Which I should study up on. Which I don't, since I don't have the need. And here I am ... vicious circle.
To be fair there is the new Kotlin based wiring API which avoids that with the caveat you need to instantiate everything manually etc. Which is probably a decent tradeoff for some folks.
I thought so too. And wrote simple web service using Helidon SE. It eats 300+ MB of RAM. I spend some time trying to optimize GC and all that stuff. Similar node service would eat 30 MB of RAM.
May be Graal would save us all. Until then Java is beyond salvation.
Other than a very very niche usecase, I really don’t see how eating 300 MB of RAM is so problematic when we quite literally have servers with terabytes of RAM. Yeah java can be configured to run GC all the time and target <100M of ram, but it rather runs the GC only seldom (jvm is actually one of the most energy efficient runtimed languages out there!) and trades memory usage to throughput.
Because in the cloud you're paying hefty price for every MB of RAM. For example with Jelastic you have 128 MB per cloudlet. And it's 2x difference between 120 MB and 130 MB. And with dedicated servers I don't have terabytes of RAM, I have two server with 24 GB each.
And no, you can't configure Java to target <100 MB of RAM. I configured it with -Xmx64m and it still eats around 300 MB. Java just fat and you can't do nothing about it at this time.
Perhaps it's easier to address the problems in a managed environment and I really do hope they pull it off. Also it's unclear whether virtual threads will support async file I/O out of the box, or ever. (C# does have Async methods on files.)
> In Java 1.1, green threads were the only threading model used by the Java virtual machine (JVM),[8] at least on Solaris. As green threads have some limitations compared to native threads, subsequent Java versions dropped them in favor of native threads.[9][10]
So is the "new" part that green threads are coming back to Java?
The "some limitations" there was mostly that original green threads implementation in Java lived on a single OS thread, so you could only use one core. Presumably that's changing this time around?
Yes, by my reading of the linked JEP, these virtual threads are executed by a pool of honest-to-goodness OS threads (so a virtual thread might pause on one thread and get resumed on another).
On Windows NT as well. On my first job around 2000 I did a little bit of Java programming. As far as I recall, JVM scheduled all their threads on top of a single OS thread.
Windows NT natively supported threads when Java was released, and Java on Windows NT (and Windows 95) in 1996 used OS threads. There were some problems when running on machines with more than one CPU, but I never figured out if this was the the fault of the OS or the JVM.
A key point that I was particularly happy about is that ThreadLocals work well with virtual threads. My personal killer application: MDC for logging. I have just tried to build proper logging into a Javascript/Typescript application and none of the logging frameworks support MDC (or need cumbersome workarounds) because the async-isms in the underlying language prevent it from working as intended.
Thread locals work, but we think in scope locals we have a better more robust mechanism more suited to the way thread locals are used in things like logging libraries.
Anyone know if this preview JEP would make it into Java 18?
We've been hearing out openjdk's project loom for a while, but we haven't gotten to try this out in Java mainline. I am guessing this will take at least two previews before an initial release. And given the speed the ecosystem moves at, we may not see this reaching widespread use for quite a while.
It's very close but I wouldn't expect it in 18, much more likely to land in 19 as preview. I have been using Loom builds for a small app for a while and haven't encountered anything strange (though last time I checked virtual threads confused the crap out of my IDE debugger) so I expect that there should be very few bugs to fix and thus should have an easy path to LTS in Java 21.
They're not coroutines though. This is a little semantic, but a coroutine normally uses cooperative multitasking exclusively.
Something like:
coroutine foo
while queue not full
put something in queue
when full
yield bar
coroutine bar
while queue not empty
take from queue
do something with what was taken
when empty
yield foo
Each time the coroutine yields, it removers it's state and execution resume another coroutine, and when execution is yield back it too resume from the yield point.
As I understand, in Java, they are not adding coroutines, but something that is a virtual thread, which is more like a green or lightweight thread. It means that it can be pre-emptively paused and resumed, it doesn't have to voluntarily yield. There is some scheduler that could decide when to execute which virtual thread and so on.
Right; Java's virtual threads are (*adjusts glasses*) preemptively-scheduled stackful continuations, meaning (1) that they execute in parallel and in cooperation with the OS' scheduler (*) (which automatically timeslices the CPU amongst threads), and (2) that each continuation is the full call stack at its yield site.
The alternative for point 1 is cooperative scheduling (announcing explicitly when they yield), as you've described.
The alternative for point 2 is "stackless" continuations, where the task yields by returning a callback describing the next step of the task -- or, equivalently, returning some data describing the state of the task, which the task's primary entry point can use to decide where to continue from. (For instance, imagine a function with a big `switch` statement that, when invoked, decides which case to jump into based on its argument, which the caller got from the return value of the last time it got invoked.) Either way, every step of the task constructs and then returns out of its call stack, which can be much more memory efficient, but is also more painful to model tasks in without help from the compiler/language.
(*) Technically, the JVM could take on the role of scheduler as well; it could count the number of bytecode instructions executed, say, and pre-empt a task after some number. This is what Lua supports for some of its sandboxing capabilities. But I think that would be counter to Java's use of an OS thread pool to execute these virtual threads; its own scheduler would be interleaving somewhat unpredictably with the OS'. You'd want to do that if you have long-running jobs that do little I/O (so they hog an OS thread)... but then you'd probably rather put those jobs on actual background worker threads.
You (or a language) don’t have to yield-to another coroutine. You may resume and yield-from as in:
coroutine producer
forever
while no full packet
resume recv into buffer
on eof return null
yield (extract packet)
coroutine consumer
while packet = (resume producer)
if packet is null break
process packet
which is more like a green or lightweight thread. It means that it can be pre-emptively paused and resumed, it doesn't have to voluntarily yield
I’ve never heard of this meaning of coro/green/light distinction. Can you please point to some literature?
In your example, it seems to still be cooperative, you simply yield to the scheduler which is itself a coroutine and will then decide what other coroutine to yield back too. Here's a naive coroutine scheduler :
ArrayList coroutines;
coroutine scheduler
for i = 0;; i = i++ % coroutines.size()
yield coroutines[i]
It's still voluntary yielding though, preemptive would be that the scheduler can at any time interupt the task, but here it can't, it will still only be possible to schedule another task ounce a yield point voluntarily yields back to the scheduler.
Actually, your example is simpler then that: (resume producer) is the same as: yield producer. And the yield with a return value is the same as: yield consumer. For the latter, the language probably allows yielding to the previous coroutine under the hood or like I said maybe it yields to a scheduler.
I was also showing that you can even do something like yield to a scheduler which will then pick the next coroutine to resume, which makes it even more "thread like", but still cooperative.
The coroutine's cooperative nature has an advantage, it naturally models coordination. With a preemptive scheme like Java virtual thread, you will still have to protect shared data and have ways to coordinate and synchronize like mutex, locks and all that.
> With a preemptive scheme like Java virtual thread, you will still have to protect shared data and have ways to coordinate and synchronize like mutex, locks and all that.
As far as I know there is nothing preventing race conditions and dead/live locks in case of coroutines either, isn’t there? Like of course if you have 1 thread these issues won’t come up, but with true parallelism, this model in itself doesn’t protect anything.
It doesn't prevent it, but it can help with synchronization.
If you have two coroutines writing to the same variable, but they yield to each other, you know they won't ever both run at the same time.
You also know if you spawn multiple coroutines that they won't yield except where they call yield, so everything before and after the yield you know will be atomic.
> If you have two coroutines writing to the same variable, but they yield to each other, you know they won't ever both run at the same time.
But that won’t be parallel just concurrent, and in case of cooperative “threads”, you could have probably written it in a more readable single threaded way, as that’s pretty much just calling two functions back and forth.
Your second point also only works when you have a single thread of execution, otherwise concurrency will entail parallelism and all the usual problems will become apparent.
Threads still have the issue of synchronization and atomicity even when only concurrent and not parallel.
That is, assuming you had a single core CPU, with threads you'd still need to synchronize things when implementing concurrency. Coroutines have a more explicit synchronization from their natural ping/pong as you yield which could be said to tend to be safer in the average case.
I think you're maybe conflating something. If two things write to the same global variable for example, that can never be parallel, but it can be concurrent. With threads, the writes to the variables need to be guarded with some synchronization mechanisms, if you forget you'll have bugs.
With coroutines, they will be naturally synchronized by the yield points.
> you could have probably written it in a more readable single threaded way, as that’s pretty much just calling two functions back and forth
It's not just calling two functions back and forth, the coroutines retain state and continue where they yielded. Each time they yield they do not consume additional stack frames.
That would be my guess as well, and I would (casually, sitting in my armchair) expect two previews for such a significant change to the JVM. I'm not expecting it to land fully until Java 20 at least -- but we should absolutely give the previews a try as they come out.
I'd love to know who, of those pinned to an LTS release, has actually made use of a support contract with a company providing contracted support for an LTS release, whether it's Oracle or another company. I don't doubt they exist, but I have no idea what that support even looks like.
My team has been happily tracking the twice-yearly JDK bumps. We started development three years ago against Java 8 and made a series of jumps (9, 11, and then 14 onward) and never really had an issue.
I'm not sure I can live without `var`, `record`, and pattern-matching `instanceof` anymore. (With `sealed` interfaces and records, the visitor pattern is long gone... I can only wait with baited breath for exhaustive pattern-matching `switch` expressions.)
I am not on a support contract (rather the opposite: small team). But I am just being careful to get caught having to spend time upgrading code that I would otherwise not have touched, just because my non-LTS JVM runs out of support. Support is not just a support contract, but also security patches being released. In my understanding, for non-LTS versions that ceases quickly when the next version is out. Particularly for non-LTS versions there may be experimental features that are not going to be compatible with subsequent versions, increasing my risk that a migration to the next version is occasionally not quickly.
pron> Assuming you've already made the last ever major upgrade past 8 (which was a relatively tough one), the reason people pay for LTS isn't because upgrades are overall cheaper -- they're costlier, actually -- but because they're willing to pay to not get new features. We've designed the LTS model mostly for legacy applications that don't see much maintenance, and want their dependencies, the JDK included, to change as little as possible.
pron> People who want a new feature to land in LTS still misunderstand what LTS is. People who upgrade from LTS to LTS every three years also misunderstand LTS, and probably get the worst of both worlds.
Personally, I only found Java 9 to be anything like a stumbling block, and that's solely because the module system (Jigsaw) threw all the tooling for a loop. You can easily avoid Jigsaw and never worry about it.
The Java folks try really hard not to break backwards compatibility in general, and modules (+ JDK internals encapsulation) are the only major bugbears to worry about. If you can upgrade, I've found it extremely worthwhile.
> Particularly for non-LTS versions there may be experimental features that are not going to be compatible with subsequent versions, increasing my risk that a migration to the next version is occasionally not quickly.
As for this, the experimental features may as well not exist if you don't enable them. You absolutely should kick the tires with them if you can, but their presence is feature-flagged off by default. I'm on a small team myself, and it's been painless for us ever since jumping to 11.
With the new oracle licensing you can use an LTS until the next LTS comes out and one additional year on top, for free. That one year should be more than enough for testing, isn’t it? Especially that thanks to strong encapsulations java updates are even more of a breeze.
Traditional wisdom says that you shouldn't use `instanceof`, because downcasting is bad and you should work with an interface uniformly implemented by any particular subclass. The Visitor pattern accomplishes this by giving the shared supertype a `match` method accepting, effectively, a bunch of callbacks, and each subclass just chooses which callback to invoke.
In algebraic type notation, this pattern replaces a function returning a sum type, X -> A + B + C, with a function accepting a callback that accepts a sum type, `X -> (A + B + C -> Y) -> Y`. But function accepting a sum type is the same as a product of functions, so you have `X -> (A -> Y, B -> Y, C -> Y) -> Y`. The product of functions is the visitor, and `X` is the thing you're visiting.
Traditional wisdom is correct when you have an open family of subclasses (i.e. you don't know, and shouldn't know, precisely how many subclasses there are). But for a closed family, it's just unnecessary; you're blinding yourself from information you already possessed.
Indeed. Since Java is moving to 2 year LTS cycle, I guess overall plan would be have it as standard feature with JDK-21 LTS. Also most likely with primitive objects and much enhanced pattern matching I feel JDK-21 is going to suck a lot oxygen from other JVM languages.
Yes, Java is finally becoming a language I don't cry myself to sleep over. I've still got other languages I (vastly) prefer, but I don't feel like I'm fighting the system nearly as much anymore. (Though, tail-call optimization would mean I don't have to obfuscate so many algorithms that are naturally recursive... it's part of Loom's charter, so maybe after virtual threads.)
I prefer Kotlin for a number of reasons but still use Java heavily as it's still a better choice in many places. I think this will be of massive benefit to both languages. Kotlin coroutines will probably be mostly relegated to multiplatform and JS backends but that is fine, server side Kotlin will take full advantage of virtual threads. :D
So the main thing is libraries. If you write your library in Kotlin then it depends on the Kotlin runtime JARs which seriously bloats it's dependencies. Not a problem where the library is only applicable in Kotlin-land but if it's generally useful from Java/Clojure/Scala then it's better to write it in Java.
The other case is general OSS software, Java reaches a wider audience in my field (distributed databases, streaming data, etc). Java is pretty much considered the lingua-franca of Big Data with some very small Scala footprint and much less fluency in Kotlin.
I generally write all my own stuff ontop of these on Kotlin but drop down into Java where I need to be able to share things.
I actually think it will greatly provide a lot of oxygen to other languages.
Virtual threads, project lilliput and valhalla is likely to be a great benefit for Clojure, which has great thread primitives and also spawn a _lot_ of objects that (mostly) don't care about identity.
> There are situations when the VM cannot suspend a virtual thread, in which case it is said to be pinned. Currently, there are two:
> When a native method is currently executing in the virtual thread (even if it is calling back into Java)
Does that mean any kind of native code is currently paying some extra cost due to the possibility of being blocking? What if I e.g. want to call a library that is known to be non-blocking, or make a syscall that is non-blocking which is not pre-wrapped by the Java standard library? E.g. a library that allows to offer interacting with a BPF map comes to my mind. Is there maybe an escape hatch for virtual thread aware java libraries, where they can tell the runtime that they want to call native code without extra guardrails and overhead?
I suspect that this is not about the possibility of blocking, but rather that native code needs a native stack (not on the Java heap) and execution state that cannot be suspended like Java code can.
If you have "short" native methods, like in a typical async I/O library, this is not a problem. They cannot be suspended while in there, but they repeatedly go back to Java code where they can be suspended.
So your scenario is only really a concern with a long-running but non-blocking native method, say, a physics library that does lots of computations. The answer is most likely: Don't run that in a virtual thread.
I/O from JNI is very rare in JVM world. I think the most common cases are going to be really CPU intensive libs like compression and encryption implementations. But yeah, just run these on a native thread pool (which you should probably do anyway).
Also all of the existing JVM IO is done with JNI. How do you think java.io is implemented itself? Of course Loom can change those implementations, but strictly speaking JNI is currently extremely common for Java IO. How big of a task supporting virtual threads for the Java libraries remains to be seen, especially for the unofficial extensions like the sun.nio.* package
> What if I e.g. want to call a library that is known to be non-blocking, or make a syscall that is non-blocking which is not pre-wrapped by the Java standard library?
My understanding is that pinning only enters the equation if you need to yield while a native frame is on the stack. If that call is non-blocking, then by definition you'll call into it and return without needing to yield the current task.
A non-blocking call should give you some way to tell when the job you've requested has completed, of course, and then you need to either poll for it or arrange to be told when it's done. You don't want to spinlock in a virtual thread (you're just hogging an OS thread continuously, which is exactly what pinning is), so either way, you'll end up blocking -- but as long as you're blocking after returning from the native call, you should be fine.
> Does that mean any kind of native code is currently paying some extra cost due to the possibility of being blocking?
I have no special insight, but I imagine any costs are only incurred if a yield actually occurs with native code on the stack. Only then would the yield logic pin the current task to the current thread.
You aren’t incurring any overhead because that native code isn’t going to try and yield execution. You would only incur the overhead if the code called back to Java, and then that Java code performed some blocking operation.
I really hope they change this `Thread.ofVirtual()` and `Thread.ofPlatform()` language; it sounds clunky and doesn't read like any Java that I've seen before.
Indeed. “of” infected Java with the awful naming conventions imposed by java.time (and to a lesser degree java.nio) because “new” is a keyword and doing “newThing” instead was not in vogue. Now it smacks of the worst kind of hipster disease and is going to enter java.lang for no good reason at all. What’s wrong with “newPlatformThread” or something similar? This shouldn’t have survived the initial sniff test.
Besides, this hasn't been targeted to a release yet, so it might not come before Java 19, which is a year from now. Even then, it will still be a preview, which likely means another year before it's a stable feature.
Does anyone know if delimited continuations are on the roadmap with this?
Potentially, virtual threads enable them (assuming a serializable environment) (co routines are also enough for this, the serialization capability is what I find interesting).
This will enable a complete freeze of an execution to be stored and even send over the network to be completed somewhere else.
The idea of serializing suspended computation across a network seems extremely attractive to a lot of people but I've never understood why it's so appealing versus the more typical approach of sending whole binaries and defining messaging protocols. Could you possibly elaborate on why this capability is exciting?
If you can just send the computation around your basically don't need the entire message/protocol boilerplate in your execution code.
For example, think of a game engine that allows to write code with loops and calling function that may wait for an event or a rule to be true. Imagine that your game can be paused stored in case of a connection lost, synced between different server or both at a server and at a client for fast response. Now without this language feature your basic game logic code gets, you can just have a wait statement inside a loop etc., How will you return to the same place on resume? You need more code, storing the entire execution state.. on every condition or a loop you either need to store something or have code that looks like a state machine etc., With this feature you don't need special design pattern, just write it, and the mess is in the language level.
The doco explains it quite well, so I won't repeat it here.
It's solving the same problem that async does, but it does it with virtual threads instead. The idea is that functions aren't coloured, and that normal threaded code will "just work".
I see some benefits of this approach, but I feel that what all of the solutions (Java, C#, Rust, etc...) are missing is structured concurrency[1], without which madness and eldritch horrors of late-night concurrent code debugging are guaranteed.
Project Loom will include structured concurrency. But most of that will come out in releases after virtual threads.
If you read the JEP though, you'll see that Executors are auto-closable now, which means you can use try-with-resources to wait for all spawned threads to stop before continuing execution.
In practice, I suspect structured concurrency will frequently requiring using the escape hatch for scenarios where a long lived background like task really is the right fit.
But the scape hatch works by.... coloring functions! Specifically if a function needs to spawn a longer lived background task, it needs to take a nursery parameter.
If a function wants to call a function that might spawn a long lived function, it needs to either own the lifetime of said nursery, or more commonly accept a nursery as a a parameter, and pass in the supplied one.
In practice with java, the nursery concept (StructuredExecutor) will only be used for those cases where it is actually helpful, (i.e. where you really want the function call to not return until all concurrent tasks are finished), and everywhere else, like background tasks, existing primitives will be used.
And all nurseries/StructuredExecutor is buying you is the ability structurally enforcing joining of the relevant virtual threads. It lets you avoid some of the common mistakes in structuring such code, but I'm not convinced that is where the eldritch horrors of concurrent code debugging live.
I think the real eldritch horrors come from buggy attempts to implement low lock code, failing to realize that locks or other synchronization is needed when accessing a certain variable, etc. Basically race condition type situations.
I personally almost never have had substantial concurrency issues related to failing to join my concurrent tasks.
In the "Alternatives" section of TFA there are some interesting comments regarding the async/await approach (which was avoided by the Virtual Threads proposal):
"Provide syntactic stackless coroutines (async/await) in the Java language. These are easier to implement than user-mode threads and would provide a unifying construct representing the context of a sequence of operations, but that construct would be new, separate from threads while being similar to them in many respect yet different in some nuanced ways, would still split the world of APIs between those designed for threads and those designed for async/await, and would require the new thread-like construct to be introduced into all aspects of the platform and its tooling, resulting in something that would take longer for the ecosystem to adopt while not being as elegant and harmonious with the platform as user-mode threads. Most languages that have chosen to adopt async/await have done so due to an inability to implement user-mode threads (Koltin), legacy semantic guarantees (JavaScript), or language-specific technical constraints (C++). These do not apply to Java."
I wonder if Rust should also be included in the final sentences.
It doesn’t block, that’s the whole point. The runtime knows if a given method is blocking or not and under the hood even blocking IO calls are implemented with non-blocking IOs. So those can be preempted.
I've watched some of Mark Rendle's talks (like this one https://youtu.be/2-mFWi5oLkM) and while watching it I realized how much I dislike when languages start to absorb ideas that are either alien to the language, or offer multiple paradigms for solving the same problems.
I can understand Java programmer want the goodies offered in languages that are more geared towards concurrency. But multiparadigmatic languages really suck because they are no longer a single language. You get islands of different practices and a partitioned set of practitioners. And they can't always use each other's code (C++ being the most extreme and painful example).
This makes me kind of glad I switched to Go 5-6 years ago. And it makes me wonder when the (good) intentions of the Go designers to not absorb every idea that comes along will go out the window and Go will start to grow knobbly bits all over.
I think you can say that for any language, other than Java. It is a deliberately slow moving language, with very few keywords and concepts going for it. It was always meant to be a simple language on top of a very high-end runtime — and indeed they manage to implement loom with no language change. Also, concurrency not being inherent to java? It was the first major language with good support for multithreading with its synchronized keyword.
Java was one step along the way, but let's say it had adequate representation of heavy-handed tools we already had in C/C++ that made some forms of concurrency somewhat easier. But it was still some ways from promoting concurrency in that threads were pretty costly and you still depended on locking to move state between threads. And it isn't like CSP hadn't been thought of.
After about 20 years of programming Java and 5-6 years programming Go I wouldn't really list concurrency as a main feature of Java. Because you kind of go at it the way you go at it in C/C++. I think someone who has programmed (for instance) Erlang would feel much the same way.
Java exposes the low-level details of parallelism, but it also allows for high level abstractions on top. Thanks to that there are indeed libraries like Akka to provide something for those that prefer the actor model, but also clojure with its immutable data structure-using concurrency and anything in between like reactive libs. Of course concurrency has never been easy in practice so the low level details are hard to get right, but they are there in a sane and easy to use way, so if not the main point of the language, I would absolutely mention it as one of the most important plus features.
Does Go have closures? Yes it does. Therefore you can start writing everything in an async style, manually passing closures around. Then build an executor interface. Voilà. Here's an utterly alien Go codebase that doesn't use green threads.
It has nothing to do with language. It has everything to do with how other libraries (especially standard libraries) structure their code.
This is literally avoiding "bifurcating the language" and avoiding the function coloring problem, its the whole point! Otherwise they could have just copied Kotlin's coroutines. Everything works as before using the familiar Thread API, just with a different underlying machinery.
your comment surprises me, because this kind of development is explicitly trying to avoid bifurcating the language. i would argue that the async io/futures libraries that exist in java are the bifurcation because programming with them is very jarring compared to threaded java.
His comment bellies his clear ignorance to both the Java development process and Loom in particular. Such comments aren't unusual where Java is concerned, it's the one language where many are willfully ignorant (well PHP is also in contention).
Pay them no mind though. Java is already the premier server side language for serious work and this is just another tool in the toolbox, hopefully I will see less RxJava in my future. :)
Yes, and I am hoping that Go won't go through the same. But if you think about how different languages have evolved they do, over time, tend to end up in places where it gets harder and harder to avoid.
It's like an implicit, magic version of them. If the implementation is perfect then it will work great and make Kotlin coroutines obsolete. If the implementation isn't perfect, everything will work great until it hits whatever corner case and blocks everything (just like when you block on a coroutine without properly shifting to a blocking executor, only less visible and harder to diagnose).
Arguably more visible because you have native JFR support for noticing such an event. Let’s remember, that Kotlin is a guest language and that the host platform implementations get host platform integrations natively.
Not only that, Kotlin has decided to marry Android as the platform language, so all JVM features not adopted by Google into ART means that Kotlin needs to decide to which master to obey, and with time Kotlin only path are to become its own platform.
Hence why JetBrains is so eager with creating duplicates from every Java library in Kotlin.
JB is investing heavily into native, JS and wasm compiler backends, so there's a counterpoint. Another is that Kotlin is steadily adding support for features in newer Java runtimes, such as records. Another is that the Kotlin/JVM backend can target up to Java 17 bytecode, which is obviously not supported by Android.
JB is trying to become Borland, with Kotlin as their Delphi while moving away from the JVM, turning Kotlin into a platform, lets see how high they fly, specially if Google's wind stops blowing.
In my experience plain old userspace code is always the easiest thing to debug, so a guest language (where a large chunk of the runtime itself will be, by definition, just plain old code) often ends up easier to understand.
In general, I would agree with you- because Kotlin or Java that corresponds to your own code should be easier to find and understand than hotspot’s C++ source code or the jit output.
Last I checked though, Kotlin was threading coroutine suspend/resume points into methods as part of bytecode generation (it’s been a while, please do let me know if I’m wrong on this) which is not something most engineers are ready to read in the simple case, much less when trying to interpret the compiler’s name mangling scheme.
In either case, the implementation that ships with the JVM will be more capable because of it’s privileged position of integrating with the runtime, so good news! Eventually Kotlin might be able to use the native facility.
Jetbrains already announced that they will be making use of this in their coroutines implementation on the JVM (they have native and js implementations as well). So, in practical terms, very little will change for Kotlin users except they gain a few additional ways to configure a co routine scope with a virtual thread pool instead of a real one when using the latest JDK that will have this. I imagine they might make using virtual threads the default for the so-called Main CoroutineScope on the JVM. Right now that would be a single native thread. So, every time you call launch or async, it might be backed by a virtual thread.
Virtual Threads are a somewhat lower level primitive than what co-routines provide. It will allow other frameworks on the JVM to integrate and benefit from it in a similar way. E.g. Vert.x, RX Java, Spring's Flux, etc. Probably using this directly is not a great idea as there are so many nice frameworks to choose from already that will protect you from doing silly things. But it is nice to have a good implementation of this built into the JVM.
Kotlin's co-routines is somewhat unique in how it can work with and seamlessly integrate code written for other concurrency and asynchronous frameworks. So technically, this is just yet another thing that they can work with. If it has something that resembles a callback, a promise, a future, etc. you simply wrap it with a suspendCoRoutine and the resulting function is a nice suspending kotlin co-routine friendly function. Kotlin's co-routine library ships with extension functions for Spring Flux, RX Java, and a few more things and it is easy to write your own ones. It's a great way of taking away the pain of using those frameworks directly. In the browser, you have similar extension functions on javascript promises, and so on.
I've been doing a lot of Browser UI programming using Kotlin JS lately. Co-routines are very nice for that. Works very similar to how I use them with Spring Boot to implement non blocking APIs. We actually share a lot of kotlin code between client and server.
Far fewer applications need this than many are led to believe. That said, this is always a welcome addition although this really doesn't add more than syntax over the stuff we've had in java.nio for a couple of decades
Yeah agreed. It is definitely very welcome syntax sugar and I'll take it over JS callback hell or async/await stuff any day. I just wanted to point out that non-blocking I/O has been available in Java for a long time so high I/O throughput applications have been possible to build in Java even though it required more mental overhead.
More mental overhead and tough ecosystem, you end up in the coloured functions problem where you choose to write non blocking code so now all your dependencies / libraries must too. Nodejs worked because it was like this from day 1 but java isn’t.
This change would make the base primitives non blocking (really cheap Blocking which is better) by default, so now you can use any~ library and it should just work. Waaay better.
Is Virtual Threads part of Project Loom? i.e We expect Project Loom to land a little bit after this or are they completely separate issues and Project Loom is still far from landing?
JEPs are the mechanism by which projects deliver changes. This, and two other JEPs form the bulk of the work done as part of Loom, along with a number of other connected JEPs mentioned in this one.
Yes. As ideas were prototyped it became clear that Fibers needed to be Threads or there would be too many rough edges regarding compatibility and concepts that developers would need to keep in their heads.
This isn't really targeted for small devices - its very very applicable with large computers in datacenters (id argue those machines will benefit even more from this). Any time you wait on I/O you potentially park an entire thread when you could be multiplexing your threads. This is a net win for almost all users of threads (except perhaps 100% cpu intensive tasks)
I'm salivating over the context switching wins, myself. I've got a simulation system that uses threads to pause and resume modeled tasks at the appropriate times relative to the simulation clock, and we're just passing a token back and forth with each step of a task. These threads aren't used for parallelism; they're just used to capture the task's continuation in a convenient way, allowing us to use a direct-style approach to modeling tasks.
The context switching sucks so much that an alternate approach, swapping threads for throwing an exception on yield and re-running the method and ignoring side-effects until we get back to the resumption point, saves us a significant amount of time.
Coroutines are usually cooperatively scheduled (that's what the `co-` is for); these are pre-emptively scheduled, first by assignment to a pool of OS threads, and secondly by the OS' own thread scheduler. "Thread" is common nomenclature for pre-emptively scheduled tasks.
"Virtual" is a little less standard, but it draws on existing patterns: they've virtualized threads in the same way that the OS virtualizes the CPU (timeslicing) and virtualizes memory (literally, virtual memory).
They are not fully preemptively scheduled though right? If I have a virtual thread that enters an infinite loop it will not be possible to reclaim its host OS thread no?
That's a good question. It's not very easy to do that with regular Java threads either, though [0], and I think everybody considers those "fully preemptively scheduled".
Preemptive scheduling is more about whether the scheduler (in this case, the OS scheduling the carrier threads) can pause a thread no matter where it is in its processing, and since virtual threads are executed by OS threads (the virtual part is just the JVM tracking the context and where to resume at), they're preemptive.
Hmm, not sure I agree. In the case of OS threads, the kernel will preempt a misbehaving thread and give the CPU to another thread according to priority. In the case where a virtual thread enters an infinite loop however there is no way for the virtual thread scheduler to do the same, unless kernel interruptions of the stuck host OS thread somehow return control to the virtual thread scheduler instead of whatever code was executing when the host thread was preempted. It's not clear to me this is really feasible though.
Yes, it's definitely possible that a spinning virtual thread will effectively pin the current OS thread. The Loom folks could keep, say, a count of how many instructions the virtual thread has executed, and pre-empt it itself if it needs to. This could fight a little with the OS scheduler, so I'm not sure what the tradeoffs are, but if you absolutely cannot allow a spinning virtual thread to effectively pin an OS thread, you'd want to do this.
If you're going to do a bunch of computation without blocking, I'm not sure virtual threads are the tool you want to apply in the first place. They're meant to provide cheap blocking and cheap context switching. If you've got a job that's more CPU-bound than I/O bound, it's probably worth using OS threads instead.
Threads as an API have proven themselves over and over to be unfit for human usage (regardless of whether they're M:N or 1:1), and higher-level async libraries have already had M:N scheduling for years. This helps absolutely nobody.
Why does Oracle keep throwing money down this pit?
I am lost here. the non-goal mentioned here are actually worth taking as goals to be fulfilled so that every other language and library ecosystem which runs on JVM will get benefited.
Non-Goals:-
It is not a goal to change the existing implementation of platform threads, that represent Operating System (OS) threads.
It is not a goal to automatically convert existing thread construction to virtual threads.
It is not a goal to change the Java Memory Model.
It is not a goal to add new inter-thread communication mechanisms.
It is not a goal to offer a new data-parallelism construct in addition to parallel streams.
- Native threads are great. They have a lot of uses, why touch them
- Automatic conversion will break a lot of stuff and eliminate some of the benefits of native threads. Being able to use an API/implementation that is *almost* the same is a huge beneift
- The memory model is a completely separate thing that Java worked on for decades. You don't want to touch that and you don't need to
- Inter-thread communication is a separate thing. There's no reason to go after that. Same is true for the data parallelism stuff.
The focus should be on hitting the 98% of what matters and getting it out without breaking everything we already have.
While automatic conversion might be off the table, any library that expects (or allows) the caller to supply the ThreadFactory used to created thread will should make switching over pretty easy. Just construct the factory with `Thread.ofVirtual().factory()`, and provide it to the library. And if it is code you own, spawning threads, then switching over should be a pretty mechanical process.
On the memory model: I'd argue that technically the memory model would be implicitly updated to treat these virtual threads the same as classic threads. Which is a very minor update. Beyond that, in order to maintain that memory model, the implementation will need to ensure proper barriers are used on pausing and resuming virtual threads, so that a virtual thread resumed on a different physical core is guaranteed to see any of its previous writes (as would be expected within a "single thread").
> It is not a goal to change the existing implementation of platform threads, that represent Operating System (OS) threads.
> It is not a goal to automatically convert existing thread construction to virtual threads.
Key word: automatically. If you want an OS thread, you should be able to get one. But if you simply spawn a task into a virtual thread, that task should behave basically the same as in a platform thread, but with less context switching overhead (and less strict claiming of OS resources).
The locus of choice is at whichever part of the code constructs the threads to begin with; if you change it to use virtual threads, you shouldn't have to change anything else.
> It is not a goal to change the Java Memory Model.
Was there something you wanted changed? I've heard that OCaml's memory model is pretty stellar, but I don't know much about it myself, and I think a memory model is a sufficiently fundamental thing that changing it might cause backcompat issues -- but it depends on the change you'd like to see.
> It is not a goal to add new inter-thread communication mechanisms.
Is there something new you'd like to see? Supposedly, the existing inter-thread communication mechanisms will work just as well for virtual threads.
> It is not a goal to offer a new data-parallelism construct in addition to parallel streams.
Virtual threads are explicitly task-parallelism constructs, so this makes sense. Maybe you would build data-parallel constructs on top of them, but it's not something the JVM would be directly responsible for here.
Non-goals does not mean that they are bad goals, only that this particular project will not try to reach them. It’s a good way to manage expectations and scope creep.
> I doubt anyone would switch to Java Virtual Threads anytime soon, unless via Kotlin.
Switching to virtual threads will, at least in the microservices I work with, involve changing a few lines (Executors.newUncachedThreadPool() -> Executors.newVirtualThreadPool()).
Switching to Kotlin coroutines will involve re-writing a large part of our codebase, which is why it hasn't been done.
It would surprise me if people didn't switch to Java Virtual Threads by the thousands when it comes out.
The Java code being called, or calling, the Kotlin coroutine code also needs to be async in order to reap all the benefits. A Java codebase doing a lot of blocking calls can't just call some random Kotlin coroutine code and expect benefits, it has to be async all the way.
>I doubt anyone would switch to Java Virtual Threads anytime soon, unless via Kotlin.
I think you're mistaken. The Java community is just so tremendously larger relative to the Kotlin one, that this will have more users within months. I really liked what Kotlin was doing, but Java since got lambdas, they have closed the biggest gaps that drove migration.
The delta between Java and Kotlin is definitely decreasing.
What Kotliners used to say about Java back then:
- no data classes. Now there's records in Java. Check.
- no lightweight threads. Boom, project Loom came in.
- no support for FP. Now there are lambdas and :: operator in Java. Also, Stream API.
- no type inference. Then came 'var'.
And some minor things that are in Java now as well: pattern matching; sealed classes as a way to get ADTs in the foreseeable future; kind of immutable collections; etc.
Yeah, you can argue that these features are not-so-native and painful to use in Java comparing to other languages. And, as one who programs in Scala, I 100% agree with you here. But you can't argue that people used to switch to Kotlin and Scala without hesitation because it was worth it summing up all the switching pros and cons. Nowadays, this is not the case anymore. You don't have to risk and adopt a new language technology stack.
Java's philosophy is always to follow. It lets other languages experiment and then it implements those features after they've been proven. It has actual backwards compatibility guarantees and it has to support whatever feature it implements for eternity.
Aside from nullable types, I don't see any other feature that Java could borrow from Kotlin today to improve. Java looks up to Scala and Clojure to get ideas, there are some strong functional programming ecosystems that were bred by these languages, like Typelevel, ZIO, actor-systems like Akka, Datalog queries, contract-based systems like spec2, Stateflow testing, matcher-combinators, generally things that Clojure does differently than the evangelical notion that type-systems are the end all be all. The rest of improvements are on the JVM, like primitive objects (value types) and generic specialization.
I'm not disagreeing with what you're saying here — at all. But what I am claiming is that when it comes to interest for and adoption of Kotlin within the JVM community that pull is smaller and shrinking. And that lambdas in Java are a big part of the reason for that.
Kotlin never had any real pull on the behemoth that is Java. It’s an insignificant language (outside android, where java is not the real deal to begin with). If anything, JS and C# and other major high level languages could hurt java, but fortunately it has been developing in a fast pace, implementing whatever is worthy and was experimented by other languages.
And even on Android, many open-source projects decide to use Java as they can have access to a wider pool of programmers (Signal, etc). C# is only decreasing in usage, the runtime is just not there, C# has always been known in the enteprise as the language that skipped leg day. It's impressive how many features they've added in a short amount of time that make sense (though some will say, even a few of the Microsoft devs, that async/await was kind of rushed), but the runtime has only received some attention in the most recent times, lagging behind Java with approximately 10 years of research. JS is just JS, hate it or love it, it is here to stay, no point in arguing about this. Put some makeup on it to make things bearable with TypeScript and that's it.
There's no point in creating a new VM, given that you can target almost everything by supporting JVM, CLR, BEAM, V8(JS transpiler) or LLVM/WASM for C-like languages. All you need is an optimized, general purpose IR that will be able to spit bytecode for each one of those and JS in the case of V8. To be fair, I don't think that a language like Kotlin will be able to accomplish this, it is just too complex. I believe that languages with small cores like Clojure with its monumental extension power through macros or Eff or Koka which "let you define advanced control abstractions, like exceptions, async/await, iterators, parsers, ambient state, or probabilistic programs, as a user library in a typed and composable way" such that it will fit runtimes. Like, provide the IR, the building blocks and let ecosystems develop. Some will argue that now you are moving the meaning of polyglot from the programming language to the libary-level, which is true, as seen in the Scala community, the divide between better-Java (Play), a different kind of OOP (Akka) a Haskell-like ecosystem (Typelevel) and an idiomatic Haskell-like ecosystem (ZIO). So you get Java, Erlang and different flavors of Haskell in one language which is Scala :). Many say that Scala is big and messy but in fact the ecosystems spawned by the core of the language made it like that. Scala's spec is smaller than Java's. I would argue that Scala is less complex than Kotlin, but this is highly opinionated.
- continuation-passing style (CPS), hand-coded or compiler
- preemptive threading / processes
In between lie various solutions, like async/await (closer to CPS), and green threads (closer to preemptive threading).
The key difference between the two ends of this spectrum is memory footprint. With CPS you manually compress state into tuned structures. With threads you store state all over the stack in very inefficient ways because all those stack frames take extra space.
Any solution towards the thread side of the spectrum will yield significantly larger memory footprints than solutions towards the CPS side of the spectrum.
On the other hand, solutions towards the CPS side of the spectrum can require writing application-specific schedulers if fairness issues arise.
On the whole, solutions on the CPS side of the spectrum are better, IMO.
Java is kinda stuck with threads, so green threads make some sense. Of course, you can get pathological issues in M:N threading, so be careful about that.