Skip to content
Snippets Groups Projects
Commit d7114c02 authored by Rainer Kartmann's avatar Rainer Kartmann
Browse files

Move doc to main page of memory system

parent 88dea7c1
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 146 additions and 144 deletions
etc/doxygen/pages/armem/existing_servers_segments.md deleted 100644 → 0
+ 0
− 89
View file @ 88dea7c1
# Existing Memory Servers and Segments {#memory_system-existing_servers_segments}
This is a list of existing memory servers, their core segments and their \ref ARON "ARON" data types.
## RobotState {#memory_system-robot_state}
* Memory Server: [`RobotStateMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/RobotStateMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------------------------------------------|
| Description | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Robot description (e.g. link to XML or URDF file) | Description of ARMAR-6 |
| Proprioception | [Proprioception](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_state/aron/Proprioception.xml) | Robot configuration and internal sensor values. | Proprioception of ARMAR-6 |
| \subpage memory_system-localization "Localization" | [Transform](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_localization/aron/Transform.xml) | Transformations between frames {world,map,robot} to retrieve global robot pose. | Transfrom from robot root frame to map frame |
Missing:
- Robot calibration: should be part of `Description` core segment
## Object
* Memory Server: [`ObjectMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ObjectMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------|
| Class | [ObjectClass](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectClass.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can` |
| Instance | [ObjectInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectInstance.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can/instance_1` |
| Attachments | [ObjectAttachment and ArticulatedObjectAttachment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/Attachment.xml) | | |
## Vision
* Memory Server: [`ArMemVisionMemory` (VisionX)](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/tree/master/source/VisionX/components/armem/ArMemVisionMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------|
| ImageRGB | [`ImageRGB`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageRGB.xml) | RGB images (mono and stereo) of a camera | image |
| ImageDepth | [`ImageDepth`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageDepth.xml) | Depth images of an RGB-D or stereo camera | image |
## Speech
* Memory Server: [`SpeechMemory` (SpeechX)](https://git.h2t.iar.kit.edu/sw/armarx/speechx/-/tree/master/source/SpeechX/components/SpeechMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-----------------------|-----------------|---------------------------------------------------------------------------|------------------|
| Command | `SpeechCommand` | Language commands from speech (topic `Speech_Commands`). | `command` |
| TextListenerInterface | `Text` | Data from `TextListenerInterface` topics. (`TextToSpeech`, `RobotAction`) | `text` |
...
## Skills
* Memory Server: [`SkillsMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/SkillsMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-------------------|--------------|-------------------------------------|------------------|
| Statechart | `Transition` | Transitions in ArmarX state charts. | tbd |
## Example
An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development.
* Memory Server: [`ExampleMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/ExampleMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------|
| ExampleData | [ExampleData](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ExampleMemory/aron/ExampleData.xml) | Some example data. | example |
## GeneralPurpose
This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server.
* Memory Server: [`GeneralPurposeMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/GeneralPurposeMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|----------------------------------------|------|-------------|------------------|
| None. You can add segments on the fly. | | | |
etc/doxygen/pages/armem/introduction.md deleted 100644 → 0
+ 0
− 53
View file @ 88dea7c1
# Introduction to the Memory System {#memory_system-introduction}
## Distributed Memory
The ArMem memory system is distributed: It consists of several **memory servers** storing data in a hierarchical
structure.
A memory server can be defined in any ArmarX package and defines which kind of data it stores
(e.g. robot state, object classes and instances, visual data, speech, ...).
As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization,
aggregation, interpolation or prediction.
As each server is a separate component describing its own structure, it is simple to add new memory servers containing
special data types, which are defined in ArmarX packages further down in the dependency chain.
All memory servers follow the same hierarchical data structure and self-describing data format.
The data structure consists of several levels, storing histories/timelines of different entities stored in different
segments (see "Memory Levels" below).
The common data format is \ref Aron, which is a self-describing, hierarchical data format,
allowing extensive introspection as well as general storage.
Each item in the memory (i.e. entries in all levels) can be identified with a **Memory ID**, which contains the keys
of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an
entity instance ID specifies the whole path to the leaf of the data structure.
Technically, each memory server is one ArmarX component implementing the `armarx::armem::server::MemoryInterface`
(comprising a `armarx::armem::server::ReadingMemoryInterface` and a `armarx::armem::server::WritingMemoryInterface`).
Memory servers register themselves in the **Memory Name System (MNS)**, where they can be found by their (semantic)
memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision").
Memory servers receive data via **commits** (via the `armarx::armem::server::WritingMemoryInterface`),
and return data according to **queries** (via the `armarx::armem::server::ReadingMemoryInterface`).
## Memory Levels
A memory server stores data in a hierarchical structure consisting of several levels:
```
- Grasping (Memory)
- Grasps (Core Segment)
- KnownObjectGraspPlanner (Provider Segment)
- my_object (Entity)
- t = 2021-03-09 14:24:20.064524 (Entity Snapshot)
- 00 (Entity Instance)
- 01 (Entity Instance)
- 02 (Entity Instance)
```
| Level | Key | Description | Examples | Implementation |
|--------------------------|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------|
| Memory | Name (String) | Semantic grouping of one or more core segments. Corresponds to one memory server. | Robot, Object, Vision | `armarx::armem::wm::Memory` |
| Core Segment | Name (String) | Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a _Core Segment Type_ which all provided data must comply to. | (Robot) Proprioception, (Object) Classes, Known (Object) Instances, ImageRGB | `armarx::armem::wm::CoreSegment` |
| Provider Segment | Name (String) | Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a _Provider Segment Type_ which extends the _Core Segment Type_. | Primesense, MyObjectLocalizer, MyGraspPlanner | `armarx::armem::wm::ProviderSegment` |
| Entity | Name (String) | A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. | `image`, `some_object`, `some_object_grasps` | `armarx::armem::wm::Entity` |
| Entity Snapshot | Timestamp (`armarx::armem::Time` aka `armarx::DateTime`) | An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. | Image, object instance, grasp hypotheses at a time *t* | `armarx::armem::wm::EntitySnapshot` |
| Entity Instance | Index (int) | One instance of the segment's ARON type. | left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) | `armarx::armem::wm::EntityInstance` |
| Entity Instance Metadata | -- | Metadata stored alongside the instance's data. | Further timesteps, confidence | `armarx::armem::wm::EntityInstanceMetadata` |
etc/doxygen/pages/armem/memory_system.md
+ 146
− 2
View file @ d7114c02
# Memory System {#memory_system} # Memory System {#memory_system}
- \subpage memory_system-introduction \tableofcontents
- \subpage memory_system-existing_servers_segments
- \subpage memory_system-how_to_create_a_new_core_segment_or_memory_server - \subpage memory_system-how_to_create_a_new_core_segment_or_memory_server
## Introduction to the Memory System {#memory_system-introduction}
### Distributed Memory
The ArMem memory system is distributed: It consists of several **memory servers** storing data in a hierarchical
structure.
A memory server can be defined in any ArmarX package and defines which kind of data it stores
(e.g. robot state, object classes and instances, visual data, speech, ...).
As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization,
aggregation, interpolation or prediction.
As each server is a separate component describing its own structure, it is simple to add new memory servers containing
special data types, which are defined in ArmarX packages further down in the dependency chain.
All memory servers follow the same hierarchical data structure and self-describing data format.
The data structure consists of several levels, storing histories/timelines of different entities stored in different
segments (see "Memory Levels" below).
The common data format is \ref Aron, which is a self-describing, hierarchical data format,
allowing extensive introspection as well as general storage.
Each item in the memory (i.e. entries in all levels) can be identified with a **Memory ID**, which contains the keys
of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an
entity instance ID specifies the whole path to the leaf of the data structure.
Technically, each memory server is one ArmarX component implementing the `armarx::armem::server::MemoryInterface`
(comprising a `armarx::armem::server::ReadingMemoryInterface` and a `armarx::armem::server::WritingMemoryInterface`).
Memory servers register themselves in the **Memory Name System (MNS)**, where they can be found by their (semantic)
memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision").
Memory servers receive data via **commits** (via the `armarx::armem::server::WritingMemoryInterface`),
and return data according to **queries** (via the `armarx::armem::server::ReadingMemoryInterface`).
### Memory Levels
A memory server stores data in a hierarchical structure consisting of several levels:
```
- Grasping (Memory)
- Grasps (Core Segment)
- KnownObjectGraspPlanner (Provider Segment)
- my_object (Entity)
- t = 2021-03-09 14:24:20.064524 (Entity Snapshot)
- 00 (Entity Instance)
- 01 (Entity Instance)
- 02 (Entity Instance)
```
| Level | Key | Description | Examples | Implementation |
|--------------------------|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------|
| Memory | Name (String) | Semantic grouping of one or more core segments. Corresponds to one memory server. | Robot, Object, Vision | `armarx::armem::wm::Memory` |
| Core Segment | Name (String) | Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a _Core Segment Type_ which all provided data must comply to. | (Robot) Proprioception, (Object) Classes, Known (Object) Instances, ImageRGB | `armarx::armem::wm::CoreSegment` |
| Provider Segment | Name (String) | Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a _Provider Segment Type_ which extends the _Core Segment Type_. | Primesense, MyObjectLocalizer, MyGraspPlanner | `armarx::armem::wm::ProviderSegment` |
| Entity | Name (String) | A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. | `image`, `some_object`, `some_object_grasps` | `armarx::armem::wm::Entity` |
| Entity Snapshot | Timestamp (`armarx::armem::Time` aka `armarx::DateTime`) | An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. | Image, object instance, grasp hypotheses at a time *t* | `armarx::armem::wm::EntitySnapshot` |
| Entity Instance | Index (int) | One instance of the segment's ARON type. | left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) | `armarx::armem::wm::EntityInstance` |
| Entity Instance Metadata | -- | Metadata stored alongside the instance's data. | Further timesteps, confidence | `armarx::armem::wm::EntityInstanceMetadata` |
## Existing Memory Servers and Segments {#memory_system-existing_servers_segments}
This is a list of existing memory servers, their core segments and their \ref ARON "ARON" data types.
### RobotState {#memory_system-robot_state}
* Memory Server: [`RobotStateMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/RobotStateMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------------------------------------------|
| Description | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Robot description (e.g. link to XML or URDF file) | Description of ARMAR-6 |
| Proprioception | [Proprioception](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_state/aron/Proprioception.xml) | Robot configuration and internal sensor values. | Proprioception of ARMAR-6 |
| \subpage memory_system-localization "Localization" | [Transform](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_localization/aron/Transform.xml) | Transformations between frames {world,map,robot} to retrieve global robot pose. | Transfrom from robot root frame to map frame |
Missing:
- Robot calibration: should be part of `Description` core segment
### Object
* Memory Server: [`ObjectMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ObjectMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------|
| Class | [ObjectClass](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectClass.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can` |
| Instance | [ObjectInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectInstance.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can/instance_1` |
| Attachments | [ObjectAttachment and ArticulatedObjectAttachment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/Attachment.xml) | | |
### Vision
* Memory Server: [`ArMemVisionMemory` (VisionX)](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/tree/master/source/VisionX/components/armem/ArMemVisionMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------|
| ImageRGB | [`ImageRGB`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageRGB.xml) | RGB images (mono and stereo) of a camera | image |
| ImageDepth | [`ImageDepth`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageDepth.xml) | Depth images of an RGB-D or stereo camera | image |
### Speech
* Memory Server: [`SpeechMemory` (SpeechX)](https://git.h2t.iar.kit.edu/sw/armarx/speechx/-/tree/master/source/SpeechX/components/SpeechMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-----------------------|-----------------|---------------------------------------------------------------------------|------------------|
| Command | `SpeechCommand` | Language commands from speech (topic `Speech_Commands`). | `command` |
| TextListenerInterface | `Text` | Data from `TextListenerInterface` topics. (`TextToSpeech`, `RobotAction`) | `text` |
...
### Skills
* Memory Server: [`SkillsMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/SkillsMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-------------------|--------------|-------------------------------------|------------------|
| Statechart | `Transition` | Transitions in ArmarX state charts. | tbd |
### Example
An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development.
* Memory Server: [`ExampleMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/ExampleMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------|
| ExampleData | [ExampleData](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ExampleMemory/aron/ExampleData.xml) | Some example data. | example |
### GeneralPurpose
This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server.
* Memory Server: [`GeneralPurposeMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/GeneralPurposeMemory)
* Core Segments:
| Core Segment Name | Type | Description | Example Entities |
|----------------------------------------|------|-------------|------------------|
| None. You can add segments on the fly. | | | |
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment