Files save on baremetal RPI #137

alloncm · 2023-09-09T12:27:38Z

This will mark the save files option in #123

* Add root_dir cache and fat cache

Also increase the stack size to 2MB

* need to clear all the logs I used for debug * finish the write file impl

hope the little optimization I dd there is fast enough, cause I failed with the actual one

The file apears in root dir but is corrupted (at least in windows)

rpi/src/drivers/fat32.rs

rpi/src/peripherals/emmc.rs

rpi/src/drivers/disk.rs

still need to update the backup FAT table

alloncm · 2023-09-25T22:54:54Z

rpi/Cargo.toml

@@ -12,6 +12,7 @@ magenboy_common = {path = "../common"}
 log = "0.4"
 cfg-if = "1"
 bitfield-struct = "0.5"
+arrayvec = {version = "0.7", default-features = false}


Why arrayvec? consider also tinyvec and smallvec or heapless to replace this and the StackString later on

arrayvec is more popular then heapless and the rest dont also offer a Stack String struct which Ill need in the near future

alloncm · 2023-09-25T22:55:31Z

rpi/src/bin/baremetal/link.ld

@@ -1,7 +1,7 @@
 /* Place _start procedure at the entry address for RPI */
 __rpi_32_phys_binary_load_addr = 0x8000;
 __isr_table_addr = 0;
-__stack_size = 0x100000; /* 1MB stack */
+__stack_size = 0x300000; /* 3MB stack */


See if I can reduce this size (just for fun though 3MB ram is a lot on RPI's)

rpi/src/peripherals/emmc.rs

Cargo.toml

rpi/src/bin/baremetal/main.rs

alloncm · 2023-09-27T19:54:05Z

rpi/src/drivers/fat32.rs

+        if let Some(existing_entry) = self.root_dir_cache.as_mut_slice().into_iter().find(|d|d.file_name == name && d.file_extension == extension){
+            if (existing_entry.size as usize) < content.len(){
+                existing_entry.file_name[0] = DELETED_DIR_ENTRY_PREFIX;
+                // TODO: mark the fat entries as free


You forgot to do this, think how to test this

rpi/src/drivers/fat32.rs

Also fixed a bunch of typos thanks to a new vscode extension

alloncm · 2023-09-30T21:01:26Z

rpi/src/drivers/mod.rs

-    let buffer = &mut *core::ptr::slice_from_raw_parts_mut(t as *mut T as *mut _, core::mem::size_of::<T>());
-    return buffer;
+pub(self) fn as_mut_buffer<'a, T>(t:&'a mut T)->&'a mut [u8]{
+    unsafe {&mut *core::ptr::slice_from_raw_parts_mut(t as *mut T as *mut _, core::mem::size_of::<T>())}


Add safety comment

Verified that this is correct using Miri

Cargo.toml

alloncm · 2023-09-30T21:05:16Z

rpi/src/drivers/fat32/mod.rs

+    }
+
+    fn arrayvec_as_buffer<'a, T, const CAP:usize>(vec:&'a ArrayVec<T, CAP>)->&'a [u8]{
+        unsafe{core::slice::from_raw_parts(vec.as_ptr() as *const u8, vec.len() * core::mem::size_of::<T>())}


Add safety comment

Verified that this is correct using Miri

The fat buffer still must be aligned for sector size

alloncm · 2023-10-03T10:53:34Z

rpi/src/drivers/fat32/fat_buffer.rs

+
+#[derive(Clone, Debug)]
+pub(super) struct FatIndex{
+    sector_number:u32,


Change to sector index

alloncm · 2023-10-04T02:54:56Z

rpi/src/drivers/mod.rs

-    let buffer = &mut *core::ptr::slice_from_raw_parts_mut(t as *mut T as *mut _, core::mem::size_of::<T>());
-    return buffer;
+pub(crate) fn as_mut_buffer<'a, T>(t:&'a mut T)->&'a mut [u8]{
+    unsafe{&mut *core::ptr::slice_from_raw_parts_mut(t as *mut T as *mut _, core::mem::size_of::<T>())}


Add safety comment

alloncm · 2023-10-04T10:43:58Z

rpi/src/drivers/fat32/fat_buffer.rs

+        self.fat_internal_index.sector_offset += FAT_ENTRY_SIZE;
+        if self.fat_internal_index.sector_offset >= SECTOR_SIZE{
+            self.fat_internal_index.sector_number += 1;
+            self.fat_internal_index.sector_offset = 0;
+        }


Extract those to another function

alloncm · 2023-10-04T14:54:46Z

rpi/src/drivers/mod.rs

-    let buffer = &mut *core::ptr::slice_from_raw_parts_mut(t as *mut T as *mut _, core::mem::size_of::<T>());
-    return buffer;
+pub(self) fn as_mut_buffer<'a, T>(t:&'a mut T)->&'a mut [u8]{
+    unsafe {&mut *core::ptr::slice_from_raw_parts_mut(t as *mut T as *mut _, core::mem::size_of::<T>())}


Verified that this is correct using Miri

alloncm · 2023-10-04T14:55:08Z

rpi/src/drivers/fat32/mod.rs

+    }
+
+    fn arrayvec_as_buffer<'a, T, const CAP:usize>(vec:&'a ArrayVec<T, CAP>)->&'a [u8]{
+        unsafe{core::slice::from_raw_parts(vec.as_ptr() as *const u8, vec.len() * core::mem::size_of::<T>())}


Verified that this is correct using Miri

alloncm · 2023-10-04T16:26:16Z

rpi/src/drivers/fat32/fat_buffer.rs

+pub enum FatSegmentState{
+    Free,
+    Allocated,
+    AllocatedEof,
+    Reserved,
+    Bad,
+}
+
+impl From<u32> for FatSegmentState{
+    fn from(value: u32) -> Self {
+        match value{
+            0 => Self::Free,
+            2..=0xFFF_FFF5 => Self::Allocated,
+            0xFFF_FFFF => Self::AllocatedEof,
+            0xFFF_FFF7 => Self::Bad,
+            _ => Self::Reserved
+        }
+    }
+}
+
+impl FatSegmentState{
+    /// Checks whether a value should be part of this segment or not
+    pub fn should_continue_segment(&self, other: &Self)->bool{
+        // AllocatedEof is should never continue segment 
+        // otherwise fallback to check raw values of the enum
+        if *self == Self::AllocatedEof || *other == Self::AllocatedEof{
+            return false;
+        }
+        return self == other;
+    }
+}
+
+#[derive(Clone, Debug)]
+pub struct FatSegment{
+    pub state:FatSegmentState,
+    pub len:u32,
+    pub start_index:u32,
+}
+
+impl FatSegment{
+    pub fn new(value:u32, start_index:u32)->Self{
+        Self { state: value.into(), len: 1, start_index}
+    }
+}


Return this to the fat module

alloncm · 2023-11-18T12:34:01Z

rpi/src/bin/baremetal/main.rs

+    let mut root_dir_offset = 0;
+    const FILES_PER_LIST:usize = 20;


Move the const to the start of the function

alloncm added 6 commits September 4, 2023 16:18

WIP add file write to the fat32 fs

fbd7a57

* Add root_dir cache and fat cache

Succesfully initing the fat32 driver caches

60246e9

Also increase the stack size to 2MB

Improve the fat32 file read performance.

e0d01d4

Trying to arrange all the fat handling

c790a88

Refactor all the fat handling

f812c76

* need to clear all the logs I used for debug * finish the write file impl

Add optimization comment

5184118

alloncm changed the title ~~Feature - Files save on baremetal RPI~~ Files save on baremetal RPI Sep 10, 2023

alloncm added the enhancement New feature or request label Sep 10, 2023

alloncm added 4 commits September 16, 2023 12:49

WIP

cc135f1

Finished the init of the fat table cacheI

24530be

hope the little optimization I dd there is fast enough, cause I failed with the actual one

Still trying to implement writing file

88db99b

WIP writing files

8d81449

The file apears in root dir but is corrupted (at least in windows)

alloncm commented Sep 18, 2023

View reviewed changes

rpi/src/drivers/fat32.rs Outdated Show resolved Hide resolved

rpi/src/drivers/fat32.rs Outdated Show resolved Hide resolved

rpi/src/drivers/fat32.rs Outdated Show resolved Hide resolved

alloncm commented Sep 19, 2023

View reviewed changes

rpi/src/peripherals/emmc.rs Outdated Show resolved Hide resolved

rpi/src/drivers/disk.rs Outdated Show resolved Hide resolved

alloncm added 5 commits September 24, 2023 12:52

WIP - Still debugging

eef0b3f

Writing and reading a file now working

2ec9255

still need to update the backup FAT table

Finish syncing all the other fats

3a42a10

Trying to implement savefiles for rpibm

809dc97

Add the support for saves on baremetal

111f63b

alloncm marked this pull request as ready for review September 25, 2023 22:43

alloncm commented Sep 27, 2023

View reviewed changes

alloncm added 3 commits September 27, 2023 23:38

Self CR clean code fixes

a30f2d7

WIP

41aeb6a

The disk operations now works with any size buffer

d570013

Also fixed a bunch of typos thanks to a new vscode extension

alloncm commented Sep 30, 2023

View reviewed changes

alloncm added 4 commits October 3, 2023 20:47

The disk changes works now

ec8825e

The fat buffer still must be aligned for sector size

Refactor write root dir to use the new disk write

22cc146

For some reason this is working now

16c4bfd

Extract FatBuffer to its own module

c735ef3

alloncm force-pushed the feature/baremetal_save branch from 463ba8c to c735ef3 Compare October 4, 2023 02:52

Remove redundant logs

75cc2d4

alloncm commented Oct 24, 2023

View reviewed changes

alloncm commented Nov 6, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Files save on baremetal RPI #137

Files save on baremetal RPI #137

alloncm commented Sep 9, 2023

alloncm Sep 25, 2023

alloncm Sep 30, 2023

alloncm Sep 25, 2023

alloncm Sep 27, 2023

alloncm Sep 30, 2023

alloncm Oct 4, 2023

alloncm Sep 30, 2023

alloncm Oct 4, 2023

alloncm Oct 3, 2023

alloncm Oct 4, 2023

alloncm Oct 4, 2023

alloncm Oct 4, 2023

alloncm Oct 4, 2023

alloncm Oct 4, 2023

alloncm Nov 18, 2023

		let mut root_dir_offset = 0;
		const FILES_PER_LIST:usize = 20;

Files save on baremetal RPI #137

Are you sure you want to change the base?

Files save on baremetal RPI #137

Conversation

alloncm commented Sep 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment