Provide formal description of DMA access synchronization

2 files changed, 155 insertions, 9 deletions

diff --git a/NOTES b/NOTES
new file mode 100644
index 0000000..9afd05a
--- /dev/null
+++ b/NOTES
@@ -0,0 +1,141 @@
+DMA Access Synchronization
+==========================
+ - At driver level, few types of buffers are supported:
+    * SIMPLE - non-reusable buffers, the use infomation can be used for cleanup
+    after crashed applications.
+    * EXCLUSIVE - reusable buffers which can be mmaped by a single appliction
+    only. There is two modes of these buffers:
+	+ Buffers in a STANDARD mode are created for a single DMA operation and
+	if such buffer is detected while trying to reuse, the last operation
+	has failed and reset is needed.
+	+ Buffers in a PERSISTENT mode are preserved between invocations of
+	control application and cleaned up only after the PERSISTENT flag is 
+	removed
+    * SHARED - reusable buffers shared by multiple processes. Not really 
+    needed at the moment.
+
+    KMEM_FLAG_HW - indicates that buffer can be used by hardware, acually this
+    means that DMA will be enabled afterwards. The driver is not able to check
+    if it really was enable and therefore will block any attempt to release 
+    buffer until KMEM_HW_FLAG is passed to kmem_free routine as well. The later
+    should only called with KMEM_HW_FLAG after the DMA engine is stopped. Then,
+    the driver can be realesd by kmem_free if ref count reaches 0.
+    
+    KMEM_FLAG_EXCLUSIVE - prevents multiple processes mmaping the buffer 
+    simultaneously. This is used to prevent multiple processes use the same
+    DMA engine at the same time.
+    
+    KMEM_FLAG_REUSE - requires reuse of existing buffer. If reusable buffer is 
+    found (non-reusable buffers, i.e. allocated without KMEM_FLAG_REUSE are
+    ignored), it is returned instead of allocation. Three types of usage 
+    counters are used. At moment of allocation, the HW reference is set if 
+    neccessary. The usage counter is increased by kmem_alloc function and
+    decreased by kmem_free. Finally, the reference is obtained at returned
+    during mmap/munmap. So, on kmem_free, we do not clean
+	a) reusable buffers with reference count above zero or hardware 
+	reference set
+	b) non-exclusive buffers with usage counter above zero (For exclusive
+	buffer the value of usage counter above zero just means that application
+        have failed without cleaning buffers first. There is no easy way to 
+        detect that for shared buffers, so it is left as manual operation in
+        this case)
+        c) any buffer if KMEM_FLAG_REUSE was provided to function (I don't have
+        a clear idea why to call it at all, but I have feeling it can be useful
+    During module unload, only buffers with references can prevent cleanup. In
+    this case the only possiblity to free the driver is to call kmem_free 
+    passing FORCE flags.
+    
+    KMEM_FLAG_PERSISTENT - if passed to allocation routine, changes mode of 
+    buffer to PERSISTENT, if passed to free routine, vice-versa changes mode
+    of buffer to NORMAL. Basically, if we call 'pci --dma-start' this flag
+    should be passed to alloc and if we call 'pci --dma-stop' it should be
+    passed to free. In other case, the flag should not be present.
+
+    If application crashed, the munmap while be still called cleaning software
+    references. However, the hardware reference will stay since it is not clear
+    if hardware channel was closed or not. To lift hardware reference, the 
+    application can be re-executed (or dma_stop called, for instance).
+    * If there is no hardware reference, the buffers will be reused by next 
+    call to application and for EXCLUSIVE buffer cleaned at the end. For SHARED
+    buffers they will be cleaned during module cleanup only (no active 
+    references).
+    * The buffer will be reused by next call which can result in wrong behaviour
+    if buffer left in incoherent stage. This should be handled on upper level.
+    
+ - At pcilib/kmem level synchronization of multiple buffers is performed
+    Inconsistent buffer types:
+     * Buffers are in PRESISTENT mode, but newly allocated, OK
+     * Buffers are reused, but are not in PERSISTENT mode (for EXCLUSIVE buffers
+     this means that application has crashed during the last execution), OK
+     * Some of buffers are reused (not just REUSABLE, but actually reused), 
+     others - not, FAIL
+     * Some of buffers are REUSABLE, others - not, FAIL
+     * Some of buffers are EXCLUSIVE, others - not, FAIL
+     * Some of buffers are PERSISTENT, others - not, FAIL
+     * Some of buffers are HW, others - not, FAIL (to simplify clean-up, 
+     even if we are going to set HW flag anyway)
+     
+    On allocation error at some of the buffer, call clean routine and
+     * Preserve HW flag if buffers hold HW reference
+     * Preserve PERSISTENT flag if buffers are in PERSISTENT mode
+     * Remove REUSE flag, we want to clean if it is allowed by current buffer
+     status
+     * EXCLUSIVE flag is not important for kmem_free routine.
+    
+ - At DMA level
+    There is 4 components of DMA access:
+    * DMA engine enabled/disabled
+    * DMA engine IRQs enabled/disabled - always enabled at startup
+    * Memory buffers
+    * Ring start/stop pointers
+    
+    To prevent multiple processes accessing DMA engine in parallel, the first
+    action is buffer initialization 
+	* Always with REUSE, EXCLUSIVE, and HW flags 
+	* Optionally with PERSISTENT flag (if DMA_PERSISTENT flag is set)
+    If another DMA app is running, the buffer allocation will fail (no dma_stop 
+    is executed in this case) 
+
+    Depending on PRESERVE flag, kmem_free will be called with REUSE flag 
+    keeping buffer in memory (this is redundant since HW flag is enough) or HW
+    flag indicating that DMA engine is stopped and buffer could be cleaned.
+    PERSISTENT flag is defined by DMA_PERSISTENT flag passed to stop routine.
+    
+    PRESERVE flag is enforced if DMA_PERSISTENT is not passed to dma_stop
+    routine and either it:
+	a) Explicitely set by DMA_PERMANENT flag passed to dma_start 
+	function 
+	b) Implicitely set if DMA engine is already enabled during dma_start, 
+	all buffers are reused, and are in persistent mode.
+    If PRESERVE flag is on, the engine will not be stopped at the end of
+    execution (and buffers will stay because of HW flag).
+    
+    If buffers are reused and in PERSISTENT mode, DMA engine was on before 
+    dma_start (we not basing on PRESERVE flag, because it can be enforced), 
+    ring pointers are calculated from LAST_BD and states of ring elements.
+    If previous application crashed (i.e. buffers may be corrupted). Two
+    cases are possible:
+    * If during the call buffers were in non-PERSISTENT mode, it can be 
+    easily detected - buffers are reused, but are not in PERSISTENT mode 
+    (or at least was not before we set them to). In this case we just 
+    reinitialize all buffers.
+    * If during the call buffers were in PERSISTENT mode, it is up to 
+    user to check their consistency and restart DMA engine.]
+    
+    IRQs are enabled and disabled at each call
+    
+Register Access Synchronization
+===============================
+ We need to serialize access to the registers by the different running 
+ applications and handle case when registers are accessed indirectly by
+ writting PCI BARs (DMA implementations, for instance).
+ * An option would be to serialize at least access to CMOSIS registers
+ which are always accessed over register functions.
+
+Register/DMA Configuration
+==========================
+ - XML description of registers
+ - Formal XML-based (or non XML-based) language for DMA implementation. 
+   a) Writting/Reading register values
+   b) Wait until <register1>=<value> on <register2>=<value> report error
+   c) ... ?
diff --git a/ToDo b/ToDo
index 1f3129c..bb2c33e 100644
--- a/ToDo
+++ b/ToDo
@@ -1,9 +1,14 @@
-1. Read registers from XML description. It also makes sense to provide
-   formal XML-based language for DMA implementation. 
-   a) Writting/Reading register values
-   b) Wait until <register1>=<value> on <register2>=<value> report error
-   c) ... ?
-2. Hint for register value representation in the bank (hex, decimal)
-3. Implement software registers
-4. Support FIFO reads/writes from/to registers
-5. Provide OR and AND operations on registers in cli
+High Priority (we would need it for IPE Camera)
+=============
+ 1. Serialize access to the registers across applications
+ 2. CMake build system
+ 
+Normal Priority (it would make just few things a bit easier)
+===============
+ 1. Implement software registers (stored in kernel-memory)
+ 2. Support FIFO reads/writes from/to registers
+ 3. Provide OR and AND operations on registers in cli
+ 
+Low Priority (only as generalization for other projects)
+============
+ 1. XML configurations describing registers (and DMA engines?)