Added internal docs and a page about the database

Author: ReCore-sys

astro.config.mjs

@@ -27,6 +27,13 @@ export default defineConfig({
 						{label: 'Comparison', slug: 'about/comparison'},
 					],
 				},
+				{
+					label: "Internal Documentation",
+					items: [
+						{label: 'Getting Started', slug: 'internal/getting_started'},
+                        {label: 'Database', slug: 'internal/database'},
+					],
+				}
 			],
 			favicon: "/favicon.ico",
 			customCss: [

package-lock.json

@@ -0,0 +1,37 @@
+---
+title: Database
+description: Docs for the database system in FerrumC.
+---
+
+FerrumC uses the LMDB database for storing data in a key-value format, allowing for fast and efficient data storage and
+retrieval. Currently, this is used exclusively for storing chunks, but in the future this will be expanded to store other
+data such as player data and entities.
+
+There are several layers of abstraction in place to make working with the database easier. <br/>
+Firstly there are some functions in `/src/lib/storage/src/lmdb.rs` that vastly simplify the process of interacting with
+the database. These function provide a simple interface for reading and writing data to the database through the use of
+functions such as `get()`, `insert()`, `delete()`, and `update()`, etc. Most of these functions work with a 128-bit key,
+a table name as a string and an array of bytes as the value. <br/>
+Inside most of these functions, tokio is used to spawn a blocking task, relegating the non-async database operations to
+a separate task to prevent blocking the main thread. This is done by using the `tokio::task::spawn_blocking()` function.
+This does require some ownership shenanigans, hence the Env being wrapped in an Arc and cloned before entering the
+blocking task. <br/>
+These functions can be used by anything, but does offer a fairly primitive interface. Code needing to interact with the
+database should have their own wrappers for handling things like caching or serializing/deserializing data.
+
+There is a second layer of abstract for chunks, found in `/src/lib/world/src/db_functions.rs`. These functions are
+specifically for handling chunk data, and provide a more high-level interface for working with chunks.
+These functions handle caching, serializing/deserializing, and converting coordinates and the dimension name into a key.
+
+The specifics of caching are covered in a separate section, but the general idea is that chunks are stored in a cache
+any time they are read or written, and are removed from the cache when a time or size cap is hit. <br/>
+Serializing is currently done using the [bitcode](https://crates.io/crates/bitcode/) crate, which is a simple and
+performant way to serialize and deserialize data. This is used to serialize the chunk data into a byte array, which is
+then stored in the database, using the aforementioned primitive functions. There is also a compression step used to
+decrease the amount of disk space a world takes up<br/>
+Generating the key is done in the `create_key()` function, which takes a dimension name, an x coordinate and a z
+coordinate and produces a 128-bit key. This key is then used to interact with the database. This key is created by first
+hashing the dimension name with [wyhash](https://crates.io/crates/wyhash) and shifting the 64 bit digest into a 128-bit
+key as the first 32 bits. The x and z coordinates are then widened to 48-bit integers and shifted into the remaining
+96 bits. This ensures that the dimension name has sufficient a keyspace to not risk hash collisions with many dimensions
+and the x and z coordinates can be sufficiently large to not limit the world size.<br/>

src/content/docs/internal/database.mdx

@@ -0,0 +1,10 @@
+---
+title: Internal documentation
+description: Internal documentation for FerrumC's codebase.
+---
+
+These pages serve as a guide to the FerrumC codebase, explaining how parts of it work and providing examples of how to
+extend it.
+
+Due to the rapidly updating codebase they may not be completely up to date, but you are welcome to suggest edits to these
+docs if you notice any discrepancies.

src/content/docs/internal/getting_started.mdx

@@ -183,7 +183,8 @@ This will create a `ferrumc` executable in the `target/release` directory.
 
     3. **Import an existing world**
 
-        Place the region files (.mca) in the folder named import then run
+        Copy the world's files into the `import` directory in the FerrumC directory. This should leave you with some
+        folders such as "region", "data" and "entities" directly inside the `import` directory.
         <Tabs syncKey="os">
             <TabItem label="Linux">
                 ~~~bash
@@ -213,15 +214,15 @@ This will create a `ferrumc` executable in the `target/release` directory.
             </TabItem>
             <TabItem label="macOS">
                 <FileTree>
-                    - import region files in here
+                    - import world files in here
                         - ...
                     - config.toml
                     - ferrumc
                 </FileTree>
             </TabItem>
             <TabItem label="Windows">
                 <FileTree>
-                    - import region files in here
+                    - import world files in here
                         - ...
                     - config.toml
                     - ferrumc.exe
@@ -266,7 +267,7 @@ This will create a `ferrumc` executable in the `target/release` directory.
         ~~~
     </TabItem>
     <TabItem label="Windows">
-        ~~~cmd
+        ~~~bash
         .\ferrumc.exe
         ~~~
     </TabItem>
@@ -285,7 +286,7 @@ You can change logging level by using ```--log=<level>```:
         ~~~
     </TabItem>
     <TabItem label="Windows">
-        ~~~cmd
+        ~~~bash
         .\ferrumc.exe --log=info #for info level logging
         ~~~
     </TabItem>
@@ -298,28 +299,6 @@ You can change logging level by using ```--log=<level>```:
     - warn (Only warnings)
     - error (Only errors)
 
-<Aside type="tip">
-    You can specify the directory to treat as the root directory (the place where the config files, data files, etc. live) by setting an environment variable `FERRUMC_ROOT` to the path of the directory.
-    <Tabs syncKey="os">
-        <TabItem label="Linux">
-            ~~~bash
-            export FERRUMC_ROOT=/home/YourUser/Documents/Code/Rust/ferrumc
-            ~~~
-        </TabItem>
-        <TabItem label="macOS">
-            ~~~bash
-            export FERRUMC_ROOT=/home/YourUser/Documents/Code/Rust/ferrumc
-            ~~~
-        </TabItem>
-        <TabItem label="Windows">
-            ~~~cmd
-            set FERRUMC_ROOT=C:\Users\YourUser\Documents\Code\Rust\ferrumc
-            ~~~
-        </TabItem>
-    </Tabs>
-    This is useful if you can't move the place the binary is executed from (`cargo run` for example).
-</Aside>
-
 
 
 ---