Merge restart into main #21

Closed
apio wants to merge 0 commits from restart into main
Owner

Let's do it!!

Let's do it!!
apio added 409 commits 2023-01-08 14:38:45 +00:00
write seems to work fine, read just hangs there, waiting.
Since we're using Rust-style integer types already, why not go all in?
I am now officially a big-boy OS developer, I read the Intel manuals for this!! (Not that the OSDev wiki doesn't have the information, but it was missing descriptions for some more obscure exceptions)
This helper returns true if the Result contains a value, or false if it doesn't.
Additionally, if it has a value, it sets the passed reference to it.
AFAIK, this is the proper naming scheme. "Pages" are virtual, and "frames" physical.
To make it clearer that this API is only intented for mapping contiguous frames of physical memory (for MMIO perhaps?)
This is the first time I've actually dropped liballoc in favor of writing my own implementation.
Usually, malloc() and such looked so complicated that I preferred to let a nice external library do the job.
But I've decided to try writing my own allocator, and now we have heap memory without any 3rd party code!
And remove a lot of testing code as well.
Those are there only for common stuff (in the Luna library) that needs an environment-agnostic way of allocating memory.
new and delete are standard, thus we should use those.
Here's one advantage of Rust over C++:
Immutability by default. In Rust, you have to mark a variable as mutable, whereas in C++ you have to mark it as immutable.

What's the problem here? Usually, most variables don't need mutability. Thus we end up with const all over the place.
Run CI on push
All checks were successful
continuous-integration/drone/push Build is passing
20eeecb3e1
Rename CI pipeline
All checks were successful
continuous-integration/drone/push Build is passing
bf7bcd7d56
Make sure the OS used in CI is Linux
All checks were successful
continuous-integration/drone/push Build is passing
eaf8a8aafe
*facepalm*

This was causing page faults when having two blocks and the first one (oldest one) being freed first.
Much better now!!
x86_64: Add a friendlier handler for page faults
All checks were successful
continuous-integration/drone/push Build is passing
8ff9cb4b96
Change safe_{sub,add,mul} so they perform the operation only once
All checks were successful
continuous-integration/drone/push Build is passing
26b44e651d
Make {add,sub,mul}_will_overflow more compiler-independent
All checks were successful
continuous-integration/drone/push Build is passing
1fa99f4f64
Move make and destroy to luna
All checks were successful
continuous-integration/drone/push Build is passing
a021e7a309
Add DoublyLinkedList data structure
All checks were successful
continuous-integration/drone/push Build is passing
87fb195202
Can be used to append an item after another one instead of at the end of the list.
With doubly linked lists, this is extremely easy to achieve (O(1)).
Make LinkedList a lot better
All checks were successful
continuous-integration/drone/push Build is passing
b5c6ae253d
Heap: Avoid magic numbers
All checks were successful
continuous-integration/drone/push Build is passing
d48eb85d1d
Make alignment a template parameter to help the compiler optimize
All checks were successful
continuous-integration/drone/push Build is passing
39b310b6b9
Change the serial log to display only milliseconds
All checks were successful
continuous-integration/drone/push Build is passing
314acbfe21
Remove unnecessary error propagation in Log.cpp
All checks were successful
continuous-integration/drone/push Build is passing
c099877c35
Serial and TextConsole always succeed, no need to act as if they could fail
Replace the _noreturn macro with the C++ native attribute [[noreturn]]
All checks were successful
continuous-integration/drone/push Build is passing
8598b1e8fc
Not all of them, but if you're measuring a size it's more appropriate to use usize than u64.
Add the nodiscard attribute to make and make_array
All checks were successful
continuous-integration/drone/push Build is passing
ee276a3a35
Update README.md
All checks were successful
continuous-integration/drone/push Build is passing
a3b69a0cfa
Remove unused includes
All checks were successful
continuous-integration/drone/push Build is passing
c2927de191
kernel: Enable -Wsign-conversion and -Wcast-align
All checks were successful
continuous-integration/drone/push Build is passing
beab3454b5
size_t -> usize
All checks were successful
continuous-integration/drone/push Build is passing
986aa01948
Merge pull request 'Add a basic scheduler with threads' (#18) from threads into restart
All checks were successful
continuous-integration/drone/push Build is passing
416e96c4d5
Reviewed-on: #18
Add support for stack traces
All checks were successful
continuous-integration/drone/push Build is passing
0d437cfcca
Add convenience functions to print the stack trace directly
All checks were successful
continuous-integration/drone/push Build is passing
c8302a4fef
Add malloc wrappers
All checks were successful
continuous-integration/drone/push Build is passing
6de7753b4c
More scope guards!!
All checks were successful
continuous-integration/drone/push Build is passing
779fda307a
Kernel: Enable -Wsign-conversion
All checks were successful
continuous-integration/drone/push Build is passing
6cee208e62
For real this time, turns out me, being dumb, added it to Luna instead of the kernel.
Remove ENONE
All checks were successful
continuous-integration/drone/pr Build is passing
b9f3d3c349
If you want to return an error without meaning, use Option.
Reviewed-on: #19
This avoids copying + we don't need to destroy the item if we move it, since it either gets emptied by moving it or doesn't have anything to destroy.
Let's make sure we explicitly tell new that we don't want exceptions
A managed String which uses RAII to free its contents. It's not a proper string though, since it's read-only.
So it's a StringView... but an owned one.

Can't be copied automatically, must be either moved or copied manually by calling clone() on it.
Convert to_dynamic_unit to OwnedStringView and rename the old variant to to_dynamic_unit_cstr
All checks were successful
continuous-integration/drone/push Build is passing
41b3c8adb2
Remove some redundant error propagation
All checks were successful
continuous-integration/drone/push Build is passing
814672c771
Why can printing to the serial port or format onto a string fail?
Even if cstyle_format returns Result<usize>, we shouldn't always follow suit.
Improve cleanup on MemoryManager failure + add methods that use KernelVM
All checks were successful
continuous-integration/drone/push Build is passing
cedcfa9c63
Convert MemoryMapIterator to Option
All checks were successful
continuous-integration/drone/push Build is passing
2e24e09146
Make all methods in OwnedStringView const
All checks were successful
continuous-integration/drone/push Build is passing
4a6c59d519
Add missing include
All checks were successful
continuous-integration/drone/push Build is passing
345e13965e
No one was noticing because everyone using OwnedStringView included Result.h before it
Rename String.h -> CString.h
All checks were successful
continuous-integration/drone/push Build is passing
59765aa334
Let's not confuse String.h with a managed string class, it's in fact the equivalent of the C stdlib's <string.h>
Print stack trace on assertion fail
All checks were successful
continuous-integration/drone/push Build is passing
b316e3b3b7
Add an Atomic class
All checks were successful
continuous-integration/drone/push Build is passing
a16f357ab9
KernelVM: Make g_used_vm atomic and g_kernelvm_bitmap a LockedValue
All checks were successful
continuous-integration/drone/push Build is passing
df9a13cbfb
Implement OwnedStringView::from_string_literal
All checks were successful
continuous-integration/drone/push Build is passing
cf3b2176f0
If we cannot fail to output, it doesn't make sense to propagate errors. So if you're SURE there are no errors, use pure_cstyle_format().
If, however, output can fail, use cstyle_format().

This has a drawback of adding quite a bit of code duplication to Format.cpp.
Some of it is dealt using templates, but some code still remains duplicate.
Add comments to Format.h
All checks were successful
continuous-integration/drone/push Build is passing
16e00bada0
Tell the compiler that string_format is a printf-style function
All checks were successful
continuous-integration/drone/push Build is passing
16954695dd
Use TypeTraits in Alignment.h to make static assertions more readable
All checks were successful
continuous-integration/drone/push Build is passing
acb0ab1cd7
asm -> asm volatile
All checks were successful
continuous-integration/drone/push Build is passing
59c9d8f119
Check for overflow/underflow in parse_signed_integer
All checks were successful
continuous-integration/drone/push Build is passing
97cb57d521
TRY(): Call release_value() instead of expect_release_value()
All checks were successful
continuous-integration/drone/push Build is passing
2bc6398d3e
LinkedList: Add a prepend() method
All checks were successful
continuous-integration/drone/push Build is passing
f5de9c5589
Make LinkedList::prepend() also set the element as the last one if we have no elements
All checks were successful
continuous-integration/drone/push Build is passing
e4b971f09c
Remove unused function definition
All checks were successful
continuous-integration/drone/push Build is passing
d5b9ff1569
Remove unnecessary std:: prefix from inside std
All checks were successful
continuous-integration/drone/push Build is passing
799a02c883
Split off arch/x86_64/CPU.cpp into various files
All checks were successful
continuous-integration/drone/push Build is passing
95b0091622
MemoryManager: Run include-what-you-use
All checks were successful
continuous-integration/drone/push Build is passing
1b867151bd
Run include-what-you-use
All checks were successful
continuous-integration/drone/push Build is passing
a89ae9bed7
Add UTF-8 support to TextConsole!!
All checks were successful
continuous-integration/drone/push Build is passing
36179155e1
Not much support, since the font only covers codepoints from U+0000 to U+00FF,
(Basic Latin & Latin Extended-A), but unprintable code-points are rendered as ONE box per code-point,
instead of multiple garbage characters.
So it's Unicode-aware, even if it can't print most characters.
I think this can go in CString.h, no need to create a separate CWChar.h or something
UTF-8 part 2: Encoding wide-character strings into UTF-8
All checks were successful
continuous-integration/drone/push Build is passing
6389099808
We now have Utf8StringEncoder and Utf8Encoder (no state this time)
TextConsole: decoder -> utf8_decoder
All checks were successful
continuous-integration/drone/push Build is passing
ffd3385d0d
Update README.md
All checks were successful
continuous-integration/drone/push Build is passing
b01878cd3c
Invented function to memcpy() with a specific size, but add a null terminator.
As long as it's in memory, we can use it :)
Kernel: Demo the initrd using TarStream
All checks were successful
continuous-integration/drone/push Build is passing
6c3b7672a0
Yes, we're using the physical address. Not optimal, this is only for demo purposes.
Make sure wchar_t is wide enough to hold all Unicode code-points
All checks were successful
continuous-integration/drone/push Build is passing
4a7e48ed5d
Too bad if it isn't.
Add todo()
All checks were successful
continuous-integration/drone/push Build is passing
cda0d49a4e
Add some more errno definitions
All checks were successful
continuous-integration/drone/push Build is passing
a08310ff5e
Lock EFIXME and error_string() behind a #define
All checks were successful
continuous-integration/drone/push Build is passing
00cf267ac7
Scheduler: Make it possible for a thread to stop existing
All checks were successful
continuous-integration/drone/push Build is passing
751377de0a
Show the date and time of build :)
All checks were successful
continuous-integration/drone/push Build is passing
a63146a798
Let's set the default (unlocked) value of Spinlock's underlying atomic to 0, so even if the constructor is not called it stays like that.
Spinlock+LockedValue: Add try_lock() methods
All checks were successful
continuous-integration/drone/push Build is passing
0bdbffe0ca
For when you want to lock a resource if possible but not get blocked if it is locked by another thread.
Make MemoryManager's scope guards more robust
All checks were successful
continuous-integration/drone/push Build is passing
60520dff4c
Add a handy consume() method to LinkedList
All checks were successful
continuous-integration/drone/push Build is passing
5b72144fac
This is a special way of iterating over the list which permits removing items while iterating.
DoublyLinkedList -> LinkedList
All checks were successful
continuous-integration/drone/push Build is passing
a11a5dec1f
Heap: SCRUB IT ALL
All checks were successful
continuous-integration/drone/push Build is passing
042f999677
If your memory is all 0xacacacac there is a big probability you haven't initialized it.
Add two FIXMEs
All checks were successful
continuous-integration/drone/push Build is passing
703c3f44fb
Log: Update formatting to make ANSI weirdness more sensible
All checks were successful
continuous-integration/drone/push Build is passing
e2a6cba3c6
Make sure isize matches ssize_t on most cases
All checks were successful
continuous-integration/drone/push Build is passing
7a2cb28475
arch/x86_64: Refactor the stack tracing code to remove duplicate code
All checks were successful
continuous-integration/drone/push Build is passing
2e8ea724a0
TextConsole: Propagate UTF-8 decoding errors
All checks were successful
continuous-integration/drone/push Build is passing
293b7b0f11
TextConsole: Add wide-character overloads for print() and println()
All checks were successful
continuous-integration/drone/push Build is passing
fcefab4383
These can't fail from UTF-8 decoding errors.
UTF-8 decoder: Error out on overlong encodings
All checks were successful
continuous-integration/drone/push Build is passing
16bf8b38ea
Use limits.h for the standard definition of WCHAR_MAX
All checks were successful
continuous-integration/drone/push Build is passing
96135ff808
Update .clang-format
All checks were successful
continuous-integration/drone/push Build is passing
a3595e71a9
Add dbgln() for the luna library
All checks were successful
continuous-integration/drone/push Build is passing
9afaad8fed
Add OwnedPtr and SharedPtr
All checks were successful
continuous-integration/drone/push Build is passing
bd4b05e534
SharedPtr: Add operator=
All checks were successful
continuous-integration/drone/push Build is passing
4fa33a9689
Fix up the header install script
All checks were successful
continuous-integration/drone/push Build is passing
913b13b380
Fix it up a bit more
All checks were successful
continuous-integration/drone/push Build is passing
34e6c05cef
Use the new toolchain location
All checks were successful
continuous-integration/drone/push Build is passing
e6384ae90d
Update README.md
All checks were successful
continuous-integration/drone/push Build is passing
74235c2c99
This function allocates a continuous range of VM and maps the physical frames passed to said VM range.
Add a global initrd TarStream to make the initial ramdisk accessible everywhere
All checks were successful
continuous-integration/drone/push Build is passing
ad0f6546d7
It's also mapped into virtual memory instead of directly going into the physical location!!
Add a simple ELF loader
All checks were successful
continuous-integration/drone/push Build is passing
f2cc797599
x86_64: Add general protection fault handler
All checks were successful
continuous-integration/drone/push Build is passing
59d69f684f
Init: Move platform_init() before MemoryManager::init()
All checks were successful
continuous-integration/drone/push Build is passing
73c58bd902
This enables NX before we actually use it.
Wasn't causing problems with KVM on, but crashed with KVM off with a 'reserved bit set' page fault.
Change heap.first().has_value() to the cheaper heap.count()
All checks were successful
continuous-integration/drone/push Build is passing
7e62ee66be
x86_64: Add a getter for the initial page directory
All checks were successful
continuous-integration/drone/push Build is passing
a1eca479d5
Heap: Log more details of blocks
All checks were successful
continuous-integration/drone/push Build is passing
ed34009b50
x86_64: Add basic keyboard support with an ASYNC DRIVER
All checks were successful
continuous-integration/drone/push Build is passing
7952d1d8a0
CMakeLists improvements
All checks were successful
continuous-integration/drone/push Build is passing
8e59a0a79f
Result: Avoid double-checking when calling through to the underlying Option
All checks were successful
continuous-integration/drone/push Build is passing
400d0395a2
Instead of showing a generic "Option::value() called on an empty Option" with no useful source location,
you will get something like "check failed: m_start_node at LinkedList.h:139"
Heap: Rewrite kmalloc to use Option<HeapBlock*> instead of nullable pointers to iterate over the heap
All checks were successful
continuous-integration/drone/push Build is passing
4081186b27
At some point, this should be done inside LinkedList itself, but we have no such thing as break in for_each().
It's iterate over everything or nothing.

This also requires operator= in Option, might be also added to Result in the future.
Result: Implement operator=()
All checks were successful
continuous-integration/drone/push Build is passing
e2e21923d7
TypeTraits: Remove const, volatile, and reference
All checks were successful
continuous-integration/drone/push Build is passing
831711ab7e
SharedPtr: adopt_shared_from_owned
All checks were successful
continuous-integration/drone/push Build is passing
df8f9c6cf6
Update year range in LICENSE :)
All checks were successful
continuous-integration/drone/push Build is passing
f0e14cf7e9
This reduces calls to kmalloc() since segment data is heap-allocated, and the segments loaded will be deleted when deleting the page directory.
This function mirrors unmap_owned_and_free_vm(), but using weak unmapping (does not free the underlying physical memory)
From a TarStream. Not optimal, but OK for the moment.
User processes need to do something, amirite?
First user process!
All checks were successful
continuous-integration/drone/push Build is passing
0ea9974512
Putting it all together, we have a user process that successfully calls sys_exit() w/o crashing.
A bit crude, should be replaced by a strdup_from_user() helper to avoid touching userspace memory directly.
But it'll do for now.
Add a syscall infrastructure (our baby program can print to the console now!)
All checks were successful
continuous-integration/drone/push Build is passing
fd8a0175d9
Switch to C for userspace, with a very bare-bones libc!!
Some checks failed
continuous-integration/drone/push Build is failing
129e3c434a
Install headers before compilation
Some checks failed
continuous-integration/drone/push Build is failing
f1923ce5bd
Depend on libc headers
Some checks failed
continuous-integration/drone/push Build is failing
08608d2344
Explicitly include system headers to make it build on CI
Some checks failed
continuous-integration/drone/push Build is failing
db1941140c
Explicitly set the link path to make it build on CI
Some checks failed
continuous-integration/drone/push Build is failing
71414c0653
Make libc depend on crt{0,i,n}.o
Some checks failed
continuous-integration/drone/push Build is failing
7acef24494
Skip these because CI needed a toolchain rebuild
All checks were successful
continuous-integration/drone/push Build is passing
30555f8e5a
Not good for W^X, but the compiler decides to squash everything into a single program header :(
Bring back the OS-Specific Toolchain on restart :^)
Some checks failed
continuous-integration/drone/push Build is failing
8838e2cf22
libc: Remove crti.o and crtn.o
Some checks failed
continuous-integration/drone/push Build is failing
bdfe7dac17
Looks like these are now provided by the compiler.
CMake: Respect the ARCH env variable
Some checks failed
continuous-integration/drone/push Build is failing
83651b4b90
Fix CI by providing a host C++ compiler
All checks were successful
continuous-integration/drone/push Build is passing
0a472000be
Update c_cpp_properties.json
All checks were successful
continuous-integration/drone/push Build is passing
65d2f1c041
More c_cpp_properties.json
All checks were successful
continuous-integration/drone/push Build is passing
50509baea6
More stdlib.h implementation + commenting + atexit()
All checks were successful
continuous-integration/drone/push Build is passing
c3653cd4e6
libc: header documentation for supported functions
All checks were successful
continuous-integration/drone/push Build is passing
367a2ce521
Implement string formatting into libc :)
All checks were successful
continuous-integration/drone/push Build is passing
80f5c790f8
Of course, using <luna/Format.h> makes it so simple. No need for duplicate code!
libc: Implement mbstowcs() using Utf8StringDecoder
Some checks failed
continuous-integration/drone/push Build is failing
b851dcf9b9
libc: Implement time() using clock_gettime().
All checks were successful
continuous-integration/drone/push Build is passing
7fb2807d0c
The cool POSIX kids use clock_gettime() now because it has NANOSECONDS (and different clocks!),
but ANSI C prefers this function.

We can still implement it based on clock_gettime(), we just have to discard the NANOSECONDS.
Implement printf()
All checks were successful
continuous-integration/drone/push Build is passing
7e377ef712
More libc commenting
All checks were successful
continuous-integration/drone/push Build is passing
b70bbc0ba6
Fix comment typo
All checks were successful
continuous-integration/drone/push Build is passing
12dc96b4a5
My bad :(
Remove the word 'header' from libc header descriptions
All checks were successful
continuous-integration/drone/push Build is passing
471103e4f3
Kernel/Scheduler: Display addresses starting with 0x when creating threads
All checks were successful
continuous-integration/drone/push Build is passing
8ee634d19b
KernelVM: Expand the available VM range
All checks were successful
continuous-integration/drone/push Build is passing
fde1727218
Kernel: Fix shadow 12GiB reserved entry when running QEMU without KVM on
All checks were successful
continuous-integration/drone/push Build is passing
0c73d69a70
Fix inspired by: https://github.com/serenityos/serenity/pull/16345
toolchain: Update to binutils 2.39
Some checks failed
continuous-integration/drone/push Build is failing
0c5a84f932
CI will fail, it's normal. Wait for me to build a new toolchain for it.
We are platform-agnostic now (kind of)
All checks were successful
continuous-integration/drone/push Build is passing
8de59138ef
Still tied to x86_64, but because there's no other platform implemented.

Also making this commit just to test CI with the new toolchain.
kernel: Turn off console logging before starting the init process
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is failing
b0e5d02c9a
apio self-assigned this 2023-01-08 14:39:01 +00:00
Author
Owner

Had to merge manually because conflicts.

Had to merge manually because conflicts.
apio closed this pull request 2023-01-08 14:46:33 +00:00
apio deleted branch restart 2023-01-08 14:47:48 +00:00
Some checks failed
continuous-integration/drone/push Build is passing
continuous-integration/drone/pr Build is failing

Pull request closed

Sign in to join this conversation.
No reviewers
No Milestone
No project
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: apio/Luna#21
No description provided.