This is an attempt to mimic low level memory constraints in dynamically typed python and then addressing some of the issues in the design of simple file system. Not tested!!
SOME PROMPT STUFF CAUSING ISSUE ON WINDOWS, works fine on linux. SERVER CAN CONNECT WITH MACHINES WITHIN SAME LAN.
Runs in three modes:
| mode | description |
|---|---|
| root | only root user can use it as personal filesystem |
| server | Multiple users can have remote access to this filesystem |
| dual | admin or root user can be using this filesystem while it is also being as server for remote users |
Access rights for each user can be managed by root (
code need slight addition for this to be functional)
Each user can be in different directory simulatneously.
It makes use of arrays and each fixed size array is treated as one memory block of hard disk, each block is of fixed siz,
total number of blocks are to be decided when you initialize this file system. It also tried to address problem of HARD LINKS and SOFT LINKS and do a TRANSLATION from logical to physical address. A table of inodes is maintained. and..
EVERYTHING IS A FILE
You can MOUNT multiple filesystems, one at a time, and changes to each filesystem can be updated in their respective .txt
file. The .txt file is used as physical drive, and start address of that drive contains MasterBlock, it contains address
of other major Block Groups including MBR and InodeTable block. Each line of text file is treated as block, and there can only be fixed number of characters in each line to mimic memory constraints, if you just updated a file and it's size
is decreased then rather then reassigning new blocks to entire file it keeps the blocks which are still pointed by file and
remove only the unused blocks, if file size is increased but there is no contigious block available then it stores new
content of file in blocks not contigious with current blocks, NOT A SINGLE CHAR SPACE IS WASTED in previous block where new data would not fit. When a memory is first time read it is empty but when when some data is written in it but the later the file which points to this data is deleted then this data stays there as Garbage Data, this feature is added to simulate real hard disk.
REPL is used to mimic CLI
let
tin= number of possible inode entries
itb= inode table block
MBk= Meta Blocks, blocks that store meta information of filesystem other then inode table blocks
fbb= free blocks boundry.
fbb= ceil((tin*int_size)/(MBksize))
As with all file systems the initialization of it requires access to the meta data of hard disk. Most critical data for a minimal filesystem is
- Total Memory of System (M)
- Total Partitions (P)
- Block Size (B)
Block size for Master Block, MBR and FB is
`MBk_size` = 6 * log2(`tin`) Bytes
In this emulator, while initialization of filesystem this information is to be provided by the user.
Once initialized the filesystem maintains the following memory structure:
This blocks contains starting and ending address of major block groups, i.e MBR, Inode block and data block.
log2(
tin) Bytes
Size always smaller then default block size (4096 B)
MBR contains the partition information and start and end address for each different disc. MBR could take variable number of blocks depending on the amount of metadata one want to store in filesystem for each partition.
d = metadata size for one logical disc
n = total discs
y = total blocks to spare for MBR
y = ceil(d*n/MBk)
memory usage = y * MBk_size
Total number of MBk required for FBB is variable just like MBR, controlled by global variable which is controlled by the amount of meta-data one is willing to add. In the minimal scenario only meta-data in this block is space separated integer values.
maximum possible integer will always be less then
tin
Could be a group/could not be a group, depending on the memory consumption.
Unlike MBR Block and FBB total number of MBk in Inode Block Group is not some programmer choice, they are fixed and depends on the M
Size of one inode entry (ies) = itb size = 128 + 2 * log2(
tin) Bits
If memeory is byte addressable then `itb` in memory would look like the illustration given below where each block is 8 Bits of size, beware...this physical memory must not be confused with logical memory block.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
0 |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ft |
8 |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| 2x ap |
24 |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| hl |
32 |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ┋ |
|¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| hl |
32+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| own |
40+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ts_c |
48+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| 2x ┋ |
64+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ts_c |
72+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ts_a |
80+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| 2x ┋ |
96+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ts_a |
104+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ts_m |
112+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| 2x ┋ |
128+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| ts_m |
136+x |¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯|
| a |
136+x+y ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
x = bytes taken by hardlink entry
y = bytes taken by address/link
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
ft = file type flag, a binary value. 1 Bit but must hold entire physical block.
ap = access permissions, 2 Bytes.
hl = hardlinks, log2(tin)/8 Bytes.
own = ownership, 1 Byte.
ts_x = ts_c, ts_a, ts_m (time stamp for created, last accessed and modified) each take 4 Bytes.
a = address/link, log2(tin)/8 Bytes.
Block size in DBG is 4096 B by default but it could be adjusted while initializing filesystem. Adjustable DBG size means it single filesystem could be used after little change to interface between storage devices of different physical block sizes for optimization purposes.
| block group | first block # | last block # |
|---|---|---|
| MBk | 0 | 0 |
| MBRB | 1 | y+1 |
| FBBG | y+2 | y+1+fbb |
| IBG | (y+2+fbb) |
tin+(2+fbb) |
python3 ./start.py
python3 ./client_cli.py
First find your PC IP address, use ifconfig in linux.
partition()
It does initialization of filesystem, you tell size of entire physical memory (in MB),
total disks you want and block size in KB. It later does calculation and generates
a schema, a design for hard disk in .txt file (hard disk) and use this as filesystem later on.
mount()
It asks for name of file system you want to mount, given the name it will load that
text file and does necessary processing for running the filesystem, once mountd then
REPL CLI will be presented and you can interact with file system
-NOTE : commands be written as it is, name and arguments be space separated,
- in below table in command column the highlited string are argumentsExcept
saveChangesall operations are either O(1) or O(N)
| Command | Description |
|---|---|
| ls | List all files within current directory |
make fname dirFlag |
Creates new file node. fname:(str) name of file you want to create. dirFlag:(0/1) 0 if you want to create a file other then a directory/folder else 1 |
delete fname |
This function deletes the named file recursively such that all children of this node are deletedfilename:(str) |
chDir node |
Switch current directory with the directory specified.node:(str) name of directory to move in.Move to parent directory: USE .. as fname to move to parent directory |
move node1 node2 |
Move file1 into file2. Recursively change pointers to file movednode1:(str) Can be a folder or a non dir file.node2:(str) Must be a dir. |
open fname mode |
Opens named file in given mode. Returns an object which is used to manipulate this file mode:(char) a/r/w Current User must be authorized to open this file in given mode. |
write fname loc text |
Writes text starting from location loc of file fnameloc:(int) index of file from where data must written. Append: Use -1 as loc to just append and avoid overwriting existing data |
read fname start end |
Reads data starting from location start of file fname upto location end of file.Read till end: Use -1 as end to read till EOF. |
truncate fname size |
Removes size number if bytes from the end of the file fname. |
close fname |
Closes the file fname if open. |
t opcode-delay argX |
Uses multithreading to perform operations on the files (non-dir files)opcode:(int)0: read1: write2: truncateargX:(list)if opcode is 0 then argX will be all the arguments that are usually given for read operation and vice versaNOTE: opcode and delay must be separated by - and opcode is must argument but delay is optional. delay decides the delay in seconds between the start of the thread and the start of operation that thread is supposed to perform. If it is a write operation on file f1 then the thread which is called after this write thread must wait atleast delay seconds in the wait queue. |
| t show | Sees if any of the thread has returned anything, if yes then show the command user have used and corresponding output from thread. |
blocksConsumed fname |
Prints the id of all blocks currently in use by named file if present in current directory Returns -1 if no block is consumed. |
data index |
Shows data blocks. index:(int) block index it can be between -1 and any positive number. |
memmap index |
Print all the inode table showing all the addresses used and all the links and pointers to the file, to understand this output you must understand schema of txt file, If index given then only that indexed inode row shownindex:(int) block index it can be between -1 and any positive number. |
| currentInode | Prints the inode current directory is using. |
| freeSpace | Prints the block ids which are not currently pointed by any file. |
| blockSize | Prints the block size in KB current file system is using. |
| logicalSpace | Logical space is the total possible number of entries in inode table. It shows total logical space. Note: if logical space is consumed you can not create new files, even though physical might be available |
| partitionInfo | Prints the partition information of the current filesystem. |
| masterBlock | Show content of MasterBlock, not the metadata of it.Major block groups are - Master Boot Record ( mbr) Boot record for File System, not system - Free Blocks ( fb) Contain unused block # to maintain a heap. - Inode Block ( ib) Contains Inode Table |
| saveChanges | If you want to close file system then don't forgot to update it on txt file. This function lets you to save your updated filesysten. You can use span a separate thread for saveing file. NOTE: You actualy need not to do a manual save, a separate process gets started whenever you make changes to the filesystem and that process is responsible to save the chages. You can control the number of changes that needs to be done before this separate process starts, Output file of this process is autoSaved |
To concurrently open two files of the same name but of different directories use
tcommand, simple open command won't work. In short you can not open a file in one direcotry and move to different directory and open different but same named file there.
Old code snippet, now you just commands mentioned above.
Below data is for my use.
Callback structure created transfers some status codes from function to function, each status code corresponds to some specific type of success or failure, action that needs to be taken is decided on the end of final callback returned status value, and this callbacks status value is in affect controlled by previous functions in this chain and so on. Using these status codes allows to take action at the end so as to avoid handling situations in the mid of function call stack.
| status code | description |
|---|---|
| 0 | success on client request, but response must not be given |
| 1 | success on client request, but response must be given |
| -1 | file system corrupted |
| -2 | Wrong arguments used |
| -3 | Command does not exist |
| -4 | file not found |
| -5 | file type mismatch |
| -6 | disk error while initialization of file system |
| -7 | Memory not available |
| -8 | Block not found |
| -9 | File can not be trimmed |
| -10 | Variant of status -5, signals that file is not a directory |
| -11 | File not found |
| -12 | Open File Table (OFT) missing pointer to file |
| -13 | File can not be put in OFT twice. |
| -14 | Integer not given |
| -15 | Index Out of Bound |
| -19 | Connection Limit Achieved |
| -20 | Already Connected |
| -21 | Socket Couldn't open |
- RN in this code, a malicious user can keep server busy by sending infinite requests from IP address already connected with server at the moment, this needs to be handled.
- Allow each user to have its separate Open File Table
- When client is already connected then its new connection is successfuly restricted but prompt about connection succes is still visible, remove it.
- Renaming functions.
- Refactoring some code.
- Startup CLI.
- Proper error coding review.
- SHOULD NOT BE ABLE TO CREATE TWO FOLDERS OF SAME NAME.
- Keep record of active client connections along with time they were connected.
- Smoothly disconnecting client with request.
- When client enters e in CLI then handle correspoding request on server side to avoid crashes.
- socket.listen(no) not listening 0 clients when no is 0.
- smoothly exit server.
- If client do not close connection, instead their is system failure on client side, then it is affecting server, so handle it.
- To allow only privileged users access to file, pass userID in Open and trwt