In this article, we shall give an update on the progress that the Managarmoperating system manufactured in the final two . 5 years, it’s been a significant ride!For readers that are unfamilar with Managarm:this is a microkernel-based OSthat supports asynchronicity through the entire entire system while also providingcompatibility with plenty of Linux software.Feel absolve to try Managarm (e.g., in a virtual machine).Our README includes a download link fora nightly image and instruction for checking out the machine.
Since our 2019 status update,Managarm has been visible at various places on the internet.Most prominently,Managarm was present at CppCon 2020 that was held virtually.Our talk targets the usage of modernC++20 within Managarm (including asynchronicity via coroutines);a recording are available on YouTube:
Weve been hosted at FOSDEM 2022, where Alexander went on the basics of IPC and the overall architecture of the machine.
Another important addition is ourhandbook that describes elements of thesystem at length (targetting both users and developers). The handbook is stillquite incomplete but regular updates should be expected later on.
Since July 2020, we’ve been focusing on the 64-bit ARM(= AArch64) port of Managarm. In itscurrent form, the port has the capacity to boot into Weston and kmscon and run somecommand-line programs in QEMU. However, a big section of our software repertoireis still untested, and the port generally continues to be work happening. We havean upcoming post that’ll be going into greater detail concerning the portingprocess, implementation details, and the obstacles we faced on the way.
Rust Support in User Space
Project member @64 has been attempting to bring Rust to Managarm. Up to now, this has contains:
- Adding information regarding our
x86_64-unknown-managarm-systemtarget to rustc (in order that it can link with this LLVM and such). This initial work was done by @avdgrinten.
- Adding bindings for mlibc in Rusts libc crate (managarm/bootstrap-managarm#96).
- Patching Rusts standard library to aid Managarm, and stubbing out functionality that people usually do not implement yet (managarm/bootstrap-managarm#96).
- Integrating cargo with this build system in order that we are able to cross-compile crates for Managarm (managarm/xbstrap#48).
- Porting ripgrep, exa and alacritty (managarm/bootstrap-managarm#102, managarm/bootstrap-managarm#180).
There’s still work to be achieved before most of Rusts standard library is fully supported within Managarm. Our next priority is upstreaming our patches back to the Rust ecosystem, since maintaining these downstream has ended up being a significant struggle.
In the long run, we wish to aid Rust drivers for Managarm. This can involve adding support for Rust inside our IPC protocol codegen tool bragi, and writing a Rust wrapper for the asynchronous syscall API.
Build Servers and xbps Packages
2 yrs ago, we deployed xbbs, a distributed build server specifically crafted for xbstrap and xbps (i.e., the package manager popularized by the Void Linux distribution). It we can effectively distribute builds of individual package of our Managarm distribution across a small number of servers, while only rebuilding elements of the distribution which have changed because the last build. Additionally, we got nearer to our goal of porting and utilizing xbps itself to control system packages inside our distribution images. You may use it today to obtain packages built by us on https://builds.managarm.org/ nevertheless, you cannot utilize it in the machine itself with xbps at this time, although work is ongoing to repair that.
The goals of the subproject include:
- Building packages as updates are pushed and checking their validity (finding SONAME changes and similar breaking issues)
- Automated detection of errors due to inconsistent/unclean build environments
- Centralizing the tracking of reproducibility of packages
- Increasing the speed of new packages and updates reaching users and reducing the opportunity of introducing new errors, by spotting them early and notifying maintainers
New features in the kernel, POSIX along with other servers
Hardware virtualization (VMX) support. Soon afterour 2019 status update,Managarm received support for hardware virtualization on IntelCPUs (using Intels VMX). We intend to extend virtualization support to AMD CPUsas well (which implement AMD SVM rather than VMX). In the long run, this willallow us to aid the KVM interface, in a way that we are able to run hypervisors likeQEMU-KVM natively on Managarm.
pthreads. As stated inour Porting Software to Managarm post,in 2020 we implemented pthread thread creation along with other related functions. mlibc has alsogotten a pthread cancellation implementation, and we’ve the next blog postgoing into detail about any of it.
A fresh IDL Compiler: bragi. In February 2020, we started focus on our very own interface description language called bragi. The goal is to replace our current protobuf usage with bragi. Though it isn’t yet fully feature-complete (we still want/need to include features like variants or inheritance), bragi has already been mature enough make it possible for us to refactor a few of our IPC protocols (namely, the POSIX, hardware, and filesystem protocols) to utilize bragi. We also started implementing new protocols (just like the ostrace one) predicated on bragi.
New Drivers: AHCI and NVMe and Storage Improvements. Managarms block driver stack received significant updates in 2021. We’ve drivers for both most significant modern block device controllers AHCI and NVMe. The AHCI driver was compiled by Matt Taylor (@64), as the NVMe driver was contributed by Jin Xue (@Jimx-). Furthermore, because of PR by Geert Custers (@geertiebear), we are able to now identify partitions by their UUIDs. This feature can make identifying the boot device better quality later on and is particularly important when running Managarm on physical hardware (rather than in a virtual machine).
Networking Improvements. Our networking stack (netserver) is now able to hook up to TCP servers over IPv4. It supports basic TCP features; however, the server side of the TCP 3-way handshake and path MTU discovery isn’t implemented yet. Once these gaps are filled, we shall have a mostly complete IPv4 stack.
New Ports and Port Updates
Within the last 2 yrs, we received lots of new and sometimes updated ports, our collection contains over 250 ports now! Most of the ports are various nice-to-have things, such as for example common UNIX utilities like
gawk, development tools like
patch and we got enough of the X11 stack ported to perform XWayland and many X based apps like
gtklife. Another noteworthy thing to say this is actually the addition of a fresh bootloader called limine, which we have now use automagically (although
grub continues to be supported at the moment and you can find no plans to eliminate that support) and the addition of a stripped down
util-linux port, which include useful utilities like
As a post without images will be boring, here are a few screenshots, to begin with is Managarm running
python.From then on, we’ve
xclock.And lastly we’ve
The street to X11
The street to X11 was a significant bumpy one, with several conditions that required digging deep in to the X11 codebase. Ultimately, the largest issues were an awful epoll bug and using abstract UNIX sockets, which were not implemented yet. With that fixed (and handful of stubbing of shared memory functions in mlibc) we could actually run the
gtk-demo demo program successfully, paving just how for many other X based programs.
Beyond XWayland, work is ongoing to also run the classic X.Org server, when using its DRM-based mode setting (rather than Westons).
The majority of the pieces essential for QEMU have been completely in place, apart from
sigaltstack and partial
mprotect support. With both these missing features implemented, we are able to run QEMU on Managarm, bringing us one step nearer to being self-hosting.
Until recently, we didn’t have any DOOM port, due to the fact we’re able to not choose which source port to utilize. We eventually decided upon dsda-doom, giving us today’s, yet vanilla DOOM experience, with extra speedrunning features as an additional benefit.
We likewise have several new and upcoming ports that people will probably show-case in a follow-up post.
What do you want to achieve within the next year?
Finish porting xbps. As stated above, considerable work went into porting the xbps package manager. As the general infrastructure, both inside Managarm and outside when it comes to an repository, are setup, even more work must actually get
xbps to operate properly. We try to implement the missing functionality soon.
Polish the port collection. Currently, we’ve plenty of ports that just work at least partially, however, many ports use pretty ugly hacks to access that state. We have to strive to obtain the quality of the ports up by improving functionality within mlibc and by detatching hacks. This includes adding better tests to check on for correctness. We also intend to start upstreaming support patches in a way that we are able to remove some patches from the collection.
Complete TTY subsystem. We currently lack or incorrectly implement many TTY subsystem features (sessions and signals, process groups, etc), which are very essential for many forms of programs in addition to day-to-day life utilizing the system. This goal can be accompanied by concluding Unix process credentials and signals.
Some remaining goals from last time include porting more software, especially to self-host, improving the blockdev stack, making the machine generally more stable and improving the netstack, especially the TCP implementation; and, needless to say, there’s always more hardware to boost support for.
We have been always searching for new contributors. If you need to try theproject, searching ourGitHub issues for interesting tasks.Apart from contributions to the kernel and/or POSIX layer,we’d be particularly thinking about network driversand drivers for additional file systems (apart from ext2),since Managarm currently lacks good drivers in these areas.