free counter
Tech

Managarm: August 2022 Update

Post byArsen Arsenovi (@ArsenArsen),Dennis Bonke (@Dennisbonke),Geert Custers (@geertiebear),Alexander van der Grinten (@avdgrinten),Matt Taylor (@64)and Kacper Somiski (@qookei).

Introduction

In this article, we shall give an update on the progress that the Managarmoperating system manufactured in the final two . 5 years, it’s been a significant ride!For readers that are unfamilar with Managarm:this is a microkernel-based OSthat supports asynchronicity through the entire entire system while also providingcompatibility with plenty of Linux software.Feel absolve to try Managarm (e.g., in a virtual machine).Our README includes a download link fora nightly image and instruction for checking out the machine.

Since our 2019 status update,Managarm has been visible at various places on the internet.Most prominently,Managarm was present at CppCon 2020 that was held virtually.Our talk targets the usage of modernC++20 within Managarm (including asynchronicity via coroutines);a recording are available on YouTube:

Additionally, the YouTube channelSystems with JT featuredour project. Browse the video for a morehands-on walk through the machine:

Weve been hosted at FOSDEM 2022, where Alexander went on the basics of IPC and the overall architecture of the machine.

Major Updates

Major updates since our last post add a 64-bit ARM port, the beginning of the RISC-V port, support for the Rust program writing language in user space, and support for the xbps package manager.

Another important addition is ourhandbook that describes elements of thesystem at length (targetting both users and developers). The handbook is stillquite incomplete but regular updates should be expected later on.

AArch64 Port

Since July 2020, we’ve been focusing on the 64-bit ARM(= AArch64) port of Managarm. In itscurrent form, the port has the capacity to boot into Weston and kmscon and run somecommand-line programs in QEMU. However, a big section of our software repertoireis still untested, and the port generally continues to be work happening. We havean upcoming post that’ll be going into greater detail concerning the portingprocess, implementation details, and the obstacles we faced on the way.

Rust Support in User Space

Project member @64 has been attempting to bring Rust to Managarm. Up to now, this has contains:

There’s still work to be achieved before most of Rusts standard library is fully supported within Managarm. Our next priority is upstreaming our patches back to the Rust ecosystem, since maintaining these downstream has ended up being a significant struggle.

In the long run, we wish to aid Rust drivers for Managarm. This can involve adding support for Rust inside our IPC protocol codegen tool bragi, and writing a Rust wrapper for the asynchronous syscall API.

Build Servers and xbps Packages

2 yrs ago, we deployed xbbs, a distributed build server specifically crafted for xbstrap and xbps (i.e., the package manager popularized by the Void Linux distribution). It we can effectively distribute builds of individual package of our Managarm distribution across a small number of servers, while only rebuilding elements of the distribution which have changed because the last build. Additionally, we got nearer to our goal of porting and utilizing xbps itself to control system packages inside our distribution images. You may use it today to obtain packages built by us on https://builds.managarm.org/ nevertheless, you cannot utilize it in the machine itself with xbps at this time, although work is ongoing to repair that.

The goals of the subproject include:

  • Building packages as updates are pushed and checking their validity (finding SONAME changes and similar breaking issues)
  • Automated detection of errors due to inconsistent/unclean build environments
  • Centralizing the tracking of reproducibility of packages
  • Increasing the speed of new packages and updates reaching users and reducing the opportunity of introducing new errors, by spotting them early and notifying maintainers

New features in the kernel, POSIX along with other servers

Hardware virtualization (VMX) support. Soon afterour 2019 status update,Managarm received support for hardware virtualization on IntelCPUs (using Intels VMX). We intend to extend virtualization support to AMD CPUsas well (which implement AMD SVM rather than VMX). In the long run, this willallow us to aid the KVM interface, in a way that we are able to run hypervisors likeQEMU-KVM natively on Managarm.

pthreads. As stated inour Porting Software to Managarm post,in 2020 we implemented pthread thread creation along with other related functions. mlibc has alsogotten a pthread cancellation implementation, and we’ve the next blog postgoing into detail about any of it.

A fresh IDL Compiler: bragi. In February 2020, we started focus on our very own interface description language called bragi. The goal is to replace our current protobuf usage with bragi. Though it isn’t yet fully feature-complete (we still want/need to include features like variants or inheritance), bragi has already been mature enough make it possible for us to refactor a few of our IPC protocols (namely, the POSIX, hardware, and filesystem protocols) to utilize bragi. We also started implementing new protocols (just like the ostrace one) predicated on bragi.

New Drivers: AHCI and NVMe and Storage Improvements. Managarms block driver stack received significant updates in 2021. We’ve drivers for both most significant modern block device controllers AHCI and NVMe. The AHCI driver was compiled by Matt Taylor (@64), as the NVMe driver was contributed by Jin Xue (@Jimx-). Furthermore, because of PR by Geert Custers (@geertiebear), we are able to now identify partitions by their UUIDs. This feature can make identifying the boot device better quality later on and is particularly important when running Managarm on physical hardware (rather than in a virtual machine).

Networking Improvements. Our networking stack (netserver) is now able to hook up to TCP servers over IPv4. It supports basic TCP features; however, the server side of the TCP 3-way handshake and path MTU discovery isn’t implemented yet. Once these gaps are filled, we shall have a mostly complete IPv4 stack.

New Ports and Port Updates

Within the last 2 yrs, we received lots of new and sometimes updated ports, our collection contains over 250 ports now! Most of the ports are various nice-to-have things, such as for example common UNIX utilities like grep, sed, findutils and gawk, development tools like python, make and patch and we got enough of the X11 stack ported to perform XWayland and many X based apps like xclock and gtklife. Another noteworthy thing to say this is actually the addition of a fresh bootloader called limine, which we have now use automagically (although grub continues to be supported at the moment and you can find no plans to eliminate that support) and the addition of a stripped down util-linux port, which include useful utilities like mount and losetup.

As a post without images will be boring, here are a few screenshots, to begin with is Managarm running python.pythonFrom then on, we’ve xclock.xclockAnd lastly we’ve exa running.exa

The street to X11

The street to X11 was a significant bumpy one, with several conditions that required digging deep in to the X11 codebase. Ultimately, the largest issues were an awful epoll bug and using abstract UNIX sockets, which were not implemented yet. With that fixed (and handful of stubbing of shared memory functions in mlibc) we could actually run the gtk-demo demo program successfully, paving just how for many other X based programs.gtk2

Beyond XWayland, work is ongoing to also run the classic X.Org server, when using its DRM-based mode setting (rather than Westons).

QEMU

The majority of the pieces essential for QEMU have been completely in place, apart from sigaltstack and partial munmap/mmap/mprotect support. With both these missing features implemented, we are able to run QEMU on Managarm, bringing us one step nearer to being self-hosting.

qemu

DOOM

Until recently, we didn’t have any DOOM port, due to the fact we’re able to not choose which source port to utilize. We eventually decided upon dsda-doom, giving us today’s, yet vanilla DOOM experience, with extra speedrunning features as an additional benefit.

doom

We likewise have several new and upcoming ports that people will probably show-case in a follow-up post.

What do you want to achieve within the next year?

Finish porting xbps. As stated above, considerable work went into porting the xbps package manager. As the general infrastructure, both inside Managarm and outside when it comes to an repository, are setup, even more work must actually get xbps to operate properly. We try to implement the missing functionality soon.

Polish the port collection. Currently, we’ve plenty of ports that just work at least partially, however, many ports use pretty ugly hacks to access that state. We have to strive to obtain the quality of the ports up by improving functionality within mlibc and by detatching hacks. This includes adding better tests to check on for correctness. We also intend to start upstreaming support patches in a way that we are able to remove some patches from the collection.

Complete TTY subsystem. We currently lack or incorrectly implement many TTY subsystem features (sessions and signals, process groups, etc), which are very essential for many forms of programs in addition to day-to-day life utilizing the system. This goal can be accompanied by concluding Unix process credentials and signals.

Some remaining goals from last time include porting more software, especially to self-host, improving the blockdev stack, making the machine generally more stable and improving the netstack, especially the TCP implementation; and, needless to say, there’s always more hardware to boost support for.

Help Wanted!

We have been always searching for new contributors. If you need to try theproject, searching ourGitHub issues for interesting tasks.Apart from contributions to the kernel and/or POSIX layer,we’d be particularly thinking about network driversand drivers for additional file systems (apart from ext2),since Managarm currently lacks good drivers in these areas.

If you need to speak to the team, you will find us onDiscord or on irc.libera.chat in the #managarmchannel.Also, if you need to support Managarm, please contemplatedonating to the project!

Acknowledgements

We thank all contributors to the Managarm,mlibc and related projects.

Read More

Related Articles

Leave a Reply

Your email address will not be published.

Back to top button

Adblock Detected

Please consider supporting us by disabling your ad blocker