From 14234ca68ea111303d5dc853717847f2a252f5f1 Mon Sep 17 00:00:00 2001 From: renehiemstra Date: Wed, 27 Nov 2024 11:04:29 +0100 Subject: [PATCH] added raii.md to documentation folder. updated content. --- docs/raii.md | 177 +++++++++++++++++++++++++++++++++++++++++++++++++++ lib/raii.md | 152 ------------------------------------------- 2 files changed, 177 insertions(+), 152 deletions(-) create mode 100644 docs/raii.md delete mode 100644 lib/raii.md diff --git a/docs/raii.md b/docs/raii.md new file mode 100644 index 00000000..39d588f2 --- /dev/null +++ b/docs/raii.md @@ -0,0 +1,177 @@ +# Deterministic Automated Resource Management + +Resource management in programming languages generally falls into one of the following categories: + +1. **Manual allocation and deallocation** +2. **Automatic garbage collection** +3. **Automatic scope-bound resource management** (commonly referred to as [RAII](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization), or *Resource Acquisition Is Initialization*). + +Traditionally, Terra has only supported manual, C-style resource management. While functional, this approach limits the full potential of Terra’s powerful metaprogramming capabilities. To address this limitation, the current implementation introduces **automated resource management**. + +--- + +## Scope-Bound Resource Management (RAII) + +The new implementation provides **scope-bound resource management (RAII)**, a method typically associated with systems programming languages like C++ and Rust. With RAII, a resource's lifecycle is tied to the stack object that manages it. When the object goes out of scope and is not explicitly returned, the associated resource is automatically destructed. + +### Examples of Resources Managed via RAII: +- Allocated heap memory +- Threads of execution +- Open sockets +- Open files +- Locked mutexes +- Disk space +- Database connections + +--- + +## Experimental Implementation Overview + +The current Terra implementation supports the **Big Three** (as described by the [Rule of Three](https://en.wikipedia.org/wiki/Rule_of_three_(C%2B%2B_programming)) in C++): + +1. **Object destruction** +2. **Copy assignment** +3. **Copy construction** + +However, **rvalue references** (introduced in C++11) are not supported in Terra. As a result, the current RAII implementation is comparable to that of **C++03** or **Rust**. + +### Key Features: +Compiler support is provided for the following methods: +```terra +A.methods.__init(self : &A) +A.methods.__dtor(self : &A) +(A or B).methods.__copy(from : &A, to : &B) +``` +These methods facilitate the implementation of smart containers and pointers, such as `std::string`, `std::vector` and `std::unique_ptr`, `std::shared_ptr`, `boost:offset_ptr` in C++. + +### Design Overview +* No Breaking Changes: This implementation does not introduce breaking changes to existing Terra code. No new keywords are required, ensuring that existing syntax remains compatible. +* Type Checking Integration: These methods are introduced during the type-checking phase (handled in terralib.lua). They can be implemented as either macros or Terra functions. +* Composability: The implementation follows simple, consistent rules that ensure smooth compatibility with existing Terra syntax for construction, casting, and function returns. +* Heap resources: Heap resources are allocated and deallocated using standard C functions like malloc and free, leaving memory allocation in the hands of the programmer. The idea here is that remaining functionality, such as allocators, are implemented as libraries. + +--- + +## Safety and Future Work + +While safety is a growing concern in programming, the current implementation has several safety challenges, similar to those in C++ or Rust's unsafe mode. + +### Future Work Includes: +1. **Library support for composable allocators**: + - Tracing or debugging allocators to detect memory leaks or other faulty behavior. +2. **Compile-time borrow checking**: + - Similar to Rust or Mojo, ensuring safer resource usage. +3. **Improved lifetime analysis**: + - Making the compiler aware of when resources (e.g., heap allocations) are introduced. + - Making the compiler aware of when resources are last used. + +--- + +## Compiler supported methods for RAII +A managed type is one that implements at least `__dtor` and optionally `__init` and `__copy` or, by induction, has fields or subfields that are of a managed type. In the following we assume `struct A` is a managed type. + +To enable RAII, import the library */lib/terralibext.t* using +```terra +require "terralibext" +``` +The compiler only checks for `__init`, `__dtor` and `__copy` in case this library is loaded. + +### Object initialization +`__init` is used to initialize managed variables: +``` +A.methods.__init(self : &A) +``` +The compiler checks for an `__init` method in any variable definition statement, without explicit initializer, and emits the call right after the variable definition, e.g. +``` +var a : A +a:__init() --generated by compiler +``` +### Copy assignment +`__copy` enables specialized copy-assignment and, combined with `__init`, copy construction. `__copy` takes two arguments, which can be different, as long as one of them is a managed type, e.g. +``` +A.metamethods.__copy(from : &A, to : &B) +``` +and / or +``` +A.metamethods.__copy(from : &B, to : &A) +``` +If `a : A` is a managed type, then the compiler will replace a regular assignment by a call to the implemented `__copy` method +``` +b = a ----> A.methods.__copy(a, b) +``` +or +``` +a = b ----> A.methods.__copy(b, a) +``` +`__copy` can be a (overloaded) terra function or a macro. + +The programmer is responsible for managing any heap resources associated with the arguments of the `__copy` method. + +### Copy construction +In object construction, `__copy` is combined with `__init` to perform copy construction. For example, +``` +var b : B = a +``` +is replaced by the following statements +``` +var b : B +b:__init() --generated by compiler if `__init` is implemented +A.methods.__copy(a, b) --generated by compiler +``` +If the right `__copy` method is not implemented but a user defined `__cast` metamethod exists that can cast one of the arguments to the correct type, then the cast is performed and then the relevant copy method is applied. + +### Object destruction +`__dtor` can be used to free heap memory +``` +A.methods.__dtor(self : &A) +``` +The implementation adds a deferred call to `__dtor ` near the end of a scope, right before a potential return statement, for all variables local to the current scope that are not returned. Hence, `__dtor` is tied to the lifetime of the object. For example, for a block of code the compiler would generate +``` +do + var x : A, y : A + ... + ... + defer x:__dtor() --generated by compiler + defer y:__dtor() --generated by compiler +end +``` +or in case of a terra function +``` +terra foo(x : A) + var y : A, z : A + ... + ... + defer z:__dtor() --generated by compiler + return y +end +``` +`__dtor` is also called before any regular assignment (if a __copy method is not implemented) to free 'old' resources. So +``` +a = b +``` +is replaced by +``` +a:__dtor() --generated by compiler +a = b +``` +## Compositional API's +If a struct has fields or subfields that are managed types, but do not implement `__init`, `__copy` or `__dtor`, then the compiler will generate default methods that inductively call existing `__init`, `__copy` or `__dtor` methods for its fields and subfields. This enables compositional API's like `vector(vector(int))` or `vector(string)`. This is implemented as an extension to *terralib.lua* in *lib/terralibext.t*. + +## Examples +The following files have been added to the terra testsuite: +* *raii.t* tests whether `__dtor`, `__init`, and `__copy` are evaluated correctly for simple datatypes. +* *raii-copyctr.t* tests `__copy`. +* *raii-copyctr-cast.t* tests the combination of `metamethods.__cast` and `__copy`. +* *raii-unique_ptr.t* tests some functionality of a unique pointer type. +* *raii-shared_ptr.t* tests some functionality of a shared pointer type. +* *raii-offset_ptr.t* tests some functionality of an offset pointer implementation, found e.g. in *boost.cpp*. +* *raii-compose.t* tests the compositional aspect. + +You can have a look there for some common code patterns. + +## Current limitations +* The implementation is not aware of when an actual heap allocation is made and therefore assumes that a managed variable always carries a heap resource. It is up to the programmer to properly initialize pointer variables to nil to avoid calling 'free' on uninitialized pointers. +* Tuple (copy) assignment (regular or using `__copy`) are prohibited by the compiler in case of managed variables. This is done to prevent memory leaks or unwanted deletions in assignments such as + ``` + a, b = b, a + ``` \ No newline at end of file diff --git a/lib/raii.md b/lib/raii.md deleted file mode 100644 index 5a777151..00000000 --- a/lib/raii.md +++ /dev/null @@ -1,152 +0,0 @@ -# RAII - Resource management -Resource acquisition is initialization ([RAII](https://en.wikipedia.org/wiki/Resource_acquisition_is_initialization)) provides a deterministic means of safe resource management. It is generally associated with systems programming languages such as *c++* and *rust*. - -In the following I summarize the experimental implementation that you can find [here](https://github.com/renehiemstra/terra/tree/raii). The socalled [Big Three](https://en.wikipedia.org/wiki/Rule_of_three_(C%2B%2B_programming)) are supported: -* object destruction -* copy assignment -* copy construction - -Terra does not support rvalue references (introduced e.g. in *c++11*), so the experimental RAII support is comparable to that of *C++03* or *rust*. - -## Feature summary -Compiler support for the following methods: -``` -A.methods.__init(self : &A) -A.methods.__dtor(self : &A) -(A or B).methods.__copy(from : &A, to : &B) -``` -These methods support the implementation of smart containers and smart pointers, like *std::string*, *std::vector* and *std::unique_ptr*, *std::shared_ptr*, *boost:offset_ptr* in C++. - -The design does not introduce any breaking changes. No new keywords are introduced. Heap resources are acquired and released using the regular *C stdlib* functions such as malloc and free, leaving memory allocation in the hands of the programmer. - -If implemented, these methods are inserted judiciously during the type checking phase, implemented in *terralib.lua*. All these metamethods can be implemented as macro's or as terra functions. - -## Compiler supported methods for RAII -A managed type is one that implements at least `__dtor` and optionally `__init` and `__copy` or, by induction, has fields or subfields that are of a managed type. In the following I assume `struct A` is a managed type. - -To enable RAII, import the library */lib/terralibext.t* using -``` - require "terralibext" -``` -The compiler only checks for `__init`, `__dtor` and `__copy` in case this libreary is loaded. - -### Object initialization -`__init` is used to initialize managed variables: -``` - A.methods.__init(self : &A) -``` -The compiler checks for an `__init` method in any variable definition statement, without explicit initializer, and emits the call right after the variable definition, e.g. -``` - var a : A - a:__init() --generated by compiler -``` -### Copy assignment -`__copy` enables specialized copy-assignment and, combined with `__init`, copy construction. `__copy` takes two arguments, which can be different, as long as one of them is a managed type, e.g. -``` - A.metamethods.__copy(from : &A, to : &B) -``` -and / or -``` - A.metamethods.__copy(from : &B, to : &A) -``` -If `a : A` is a managed type, then the compiler will replace a regular assignment by a call to the implemented `__copy` method -``` - b = a ----> A.methods.__copy(a, b) -``` -or -``` - a = b ----> A.methods.__copy(b, a) -``` -`__copy` can be a (overloaded) terra function or a macro. - -The programmer is responsable for managing any heap resources associated with the arguments of the `__copy` method. - -### Copy construction -In object construction, `__copy` is combined with `__init` to perform copy construction. For example, -``` - var b : B = a -``` -is replaced by the following statements -``` - var b : B - b:__init() --generated by compiler if `__init` is implemented - A.methods.__copy(a, b) --generated by compiler -``` -If the right `__copy` method is not implemented but a user defined `__cast` metamethod exists that can cast one of the arguments to the correct type, then the cast is performed and then the relevant copy method is applied. - -### Object destruction -`__dtor` can be used to free heap memory -``` - A.methods.__dtor(self : &A) -``` -The implementation adds a deferred call to `__dtor ` near the end of a scope, right before a potential return statement, for all variables local to the current scope that are not returned. Hence, `__dtor` is tied to the lifetime of the object. For example, for a block of code the compiler would generate -``` -do - var x : A, y : A - ... - ... - defer x:__dtor() --generated by compiler - defer y:__dtor() --generated by compiler -end -``` -or in case of a terra function -``` -terra foo(x : A) - var y : A, z : A - ... - ... - defer z:__dtor() --generated by compiler - return y -end -``` -`__dtor` is also called before any regular assignment (if a __copy method is not implemented) to free 'old' resources. So -``` - a = b -``` -is replaced by -``` - a:__dtor() --generated by compiler - a = b -``` -## Compositional API's -If a struct has fields or subfields that are managed types, but do not implement `__init`, `__copy` or `__dtor`, then the compiler will generate default methods that inductively call existing `__init`, `__copy` or `__dtor` methods for its fields and subfields. This enables compositional API's like `vector(vector(int))` or `vector(string)`. This is implemented as an extension to *terralib.lua* in *lib/terralibext.t*. - -## Examples -The following files have been added to the terra testsuite: -* *raii.t* tests whether `__dtor`, `__init`, and `__copy` are evaluated correctly for simple datatypes. -* *raii-copyctr-cast.t* tests the combination of `metamethods.__cast` and `__copy`. -* *raii-unique_ptr.t* tests some functionality of a unique pointer type. -* *raii-shared_ptr.t* tests some functionality of a shared pointer type. -* *raii-offset_ptr.t* tests some functionality of an offset pointer implementation, found e.g. in *boost.cpp*. -* *raii-compose.t* tests the compositional aspect. - -You can have a look there for some common code patterns. - -## Current limitations -* The implementation is not aware of when an actual heap allocation is made and therefore assumes that a managed variable always carries a heap resource. It is up to the programmer to properly initialize pointer variables to nil to avoid calling 'free' on uninitialized pointers. -* Tuple (copy) assignment (regular or using `__copy`) are prohibited by the compiler in case of managed variables. This is done to prevent memory leaks or unwanted deletions in assignments such as -``` - a, b = b, a -``` -* Currently, there is no way to prevent unwanted calls to `__dtor` in cases such as the following. Consider -``` -terra foo() - var b : A - return bar(b) -end -``` -which will get expanded to -``` -terra foo() - var b : A - defer b:__dtor() --generated by compiler - return bar(b) -end -``` -If `bar` would return `b` then its associated heap resources would be released before they can be used in the outer scope. - -## Roadmap -The current implementation already works in a range of important applications, as can be seen in the tests and the examples above. To remove the noted limitations and to enable graceful compile-time errors my plan is to: -* support *affine types* (similar to *rust*) by checking, at compile time, that a managed variable is used (passed by value) not more than once. This essentially means that the variable is moved from (not copied) on every use. This is not restrictive, since in general you would pass managed objects by reference, not by value. -* support borrow checking (similar to *rust*) by counting, at compile time, the number of references. -* introduce a `__new` method that signals a heap allocation. This way the compiler is made aware of all heap allocations being made. \ No newline at end of file