diff --git a/docs/armem/README.md b/docs/armem/README.md deleted file mode 100644 index 1742eec0abd08fe86b0724828722338718348fd5..0000000000000000000000000000000000000000 --- a/docs/armem/README.md +++ /dev/null @@ -1,3 +0,0 @@ -* [Introduction](introduction) -* [Existing Memories (Memory Servers)](existing_memory_servers_and_segments) -* [How to create a new Core Segment or Memory Server](how_to_create_a_new_core_segment_or_memory_server) diff --git a/docs/armem/existing_memory_servers_and_segments/README.md b/docs/armem/existing_memory_servers_and_segments/README.md deleted file mode 100644 index edc04ab4a3eda6652d59f2356687aa70d34a3db1..0000000000000000000000000000000000000000 --- a/docs/armem/existing_memory_servers_and_segments/README.md +++ /dev/null @@ -1,96 +0,0 @@ -This is a list of existing memory servers, their core segments and their ARON data types. - -# `RobotState` - -* Memory Server: [`RobotStateMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/RobotStateMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|-------------------------| -| `Description` | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Robot description (e.g. link to XML or URDF file) | | -| `Proprioception` | ToDo | Robot configuration and internal sensor values. | tbd | -| [`Localization`](RobotState/Localization) | [Transform](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_localization/aron/Transform.xml) | Transformations between frames {world,map,robot} to retrieve global robot pose. | armarx::aron::Transform | - -Missing: -- Robot calibration: should be part of `Description` core segment - -# `Object` - -* Memory Server: [`ObjectMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ObjectMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------| -| `Class` | [ObjectClass](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectClass.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can` | -| `Instance` | [ObjectInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectInstance.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can/instance_1` | -| `ArticulatedObjectClass` | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | | -| `ArticulatedObjectInstance` | [Robot](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/Robot.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | | -| `Attachments` | [ObjectAttachment and ArticulatedObjectAttachment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/Attachment.xml) | | | - - -# `Vision` - -* Memory Server: [`ArMemVisionMemory` (VisionX)](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/tree/master/source/VisionX/components/armem/ArMemVisionMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------| -| `ImageRGB` | [`ImageRGB`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageRGB.xml) | RGB images (mono and stereo) of a camera | image | -| `ImageDepth` | [`ImageDepth`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageDepth.xml) | Depth images of an RGB-D or stereo camera | image | - - - -# `Speech` - -* Memory Server: [`SpeechMemory` (SpeechX)](https://git.h2t.iar.kit.edu/sw/armarx/speechx/-/tree/master/source/SpeechX/components/SpeechMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------------|-----------------|---------------------------------------------------------------------------|------------------| -| `Command` | `SpeechCommand` | Language commands from speech (topic `Speech_Commands`). | `command` | -| `TextListenerInterface` | `Text` | Data from `TextListenerInterface` topics. (`TextToSpeech`, `RobotAction`) | `text` | - -... - - -# `Skills` - -* Memory Server: [`SkillsMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/SkillsMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------|--------------|-------------------------------------|------------------| -| `Statechart` | `Transition` | Transitions in ArmarX state charts. | tbd | - - -# `GeneralPurpose` - -This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server. - -* Memory Server: [`GeneralPurposeMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/GeneralPurposeMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|----------------------------------------|------|-------------|------------------| -| None. You can add segments on the fly. | | | | - -# `Mapping` - -* Memory Server: `ToDo` -* Core Segments: - -| Core Segment Name | <br/>Type | Description | Example Entities | -|---------------------------------|-----------|--------------------------------------------------|------------------| -| [`Mapping`](RobotState/Mapping) | `ToDo` | Mapping related sensor data (e.g. point clouds). | tbd | - - -# `Example` - -An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development. - -* Memory Server: [`ExampleMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/ExampleMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------| -| `ExampleData` | [ExampleData](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ExampleMemory/aron/ExampleData.xml) | Some example data. | example | \ No newline at end of file diff --git a/docs/armem/existing_memory_servers_and_segments/RobotState/Localization/README.md b/docs/armem/existing_memory_servers_and_segments/RobotState/Localization/README.md deleted file mode 100644 index fe8ac7af138bca553c7a80c3683ff74db0156c90..0000000000000000000000000000000000000000 --- a/docs/armem/existing_memory_servers_and_segments/RobotState/Localization/README.md +++ /dev/null @@ -1,36 +0,0 @@ -# The localization segment within the RobotState memory - -## Concept - -There exist various frames that are needed to obtain the robot's global pose (pose within in the global frame): - -* [global] the "world" frame. -* [map] During mapping, the robot does not have knowledge about the world but creates a local map (with "map" as root frame). All map features (e.g. point clouds, occupancy grids, ...) are linked to this frame. -* [odom] The odometry as an integration of the robot's velocity. -* [robot] The robot's root frame - - - -For each transformation between frames, there exist individual components: - -* [global->map] **Map registration:** given a scene (objects with known poses, e.g. walls, ...), this component obtains a static transformation between a the scene and the map. -* [map->odom] **Localization** The localization component computes the transformation [map->robot] and performs map corrections by updating [map->odom]. -* [odom->robot] **Odometry** - - -### Memory design - -Within the core segment, each robot has its own provider segment (here: Armar6). Within the provider segment, entities describe the transformations between the frames ("Global,Map" -> [global->map]). - - - -### Using the localization memory - -See **RobotAPI/libraries/armem_robot_localization** - -There exist two classes: - -* TransformWriter to send transformations to memory -* TransformReader to obtain transformations between any of the frames; can also be used to obtain the "robot's global pose" - -For an example, see RobotComponents/components/carographer_mapping_and_localization (currently branch feature/cartographer_integration) diff --git a/docs/armem/existing_memory_servers_and_segments/RobotState/Mapping/README.md b/docs/armem/existing_memory_servers_and_segments/RobotState/Mapping/README.md deleted file mode 100644 index 167e058d44b78ebec0f59d97c2dc4f7ffc7eef72..0000000000000000000000000000000000000000 --- a/docs/armem/existing_memory_servers_and_segments/RobotState/Mapping/README.md +++ /dev/null @@ -1,9 +0,0 @@ -# The mapping segment within the RobotState memory - -## Concept - -Within the memory segment, sensor data is stored that is relevant for -* mapping -* map registration - - diff --git a/docs/armem/existing_memory_servers_and_segments/RobotState/Mapping/res/transformations_frames-Page-2.png b/docs/armem/existing_memory_servers_and_segments/RobotState/Mapping/res/transformations_frames-Page-2.png deleted file mode 100644 index 65e856849ff9fc62016d27b1631f8d456b5a22b1..0000000000000000000000000000000000000000 Binary files a/docs/armem/existing_memory_servers_and_segments/RobotState/Mapping/res/transformations_frames-Page-2.png and /dev/null differ diff --git a/docs/armem/introduction/README.md b/docs/armem/introduction/README.md deleted file mode 100644 index 43f2e3ac5e1b169c900f5793875bc5788a79c37a..0000000000000000000000000000000000000000 --- a/docs/armem/introduction/README.md +++ /dev/null @@ -1,42 +0,0 @@ -# Distributed Memory - -The ArMem memory system is distributed: It is comprised of several **memory servers** storing data in a hierarchical structure. -A memory server can be defined in any ArmarX package and defines which kind of data it stores (e.g. robot state, object classes and instances, visual data, speech, ...). -As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization, aggregation, interpolation or prediction. -As each server is a separate component describing its own structure, it is simple to add new memory servers containing special data types, which are defined in ArmarX packages further down in the dependency chain. - -All memory servers follow the same hierarchical data structure and self-describing data format. -The data structure consists of several levels, storing histories/timelines of different entities stored in different segments (see "Memory Levels" below). -The common data format is ARON (ArmarX Object Notation), which is a self-describing, hierarchical data format, allowing extensive introspection as well as general storage. -Each item in the memory (i.e. entries in all levels) can be identified with a **Memory ID**, which contains the keys of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an entity instance ID specifies the whole path to the leaf of the data structure. - - -Technically, each memory server is one ArmarX component implementing the `armem::server::MemoryInterface` (comprising a `ReadingMemoryInterface` and a `WritingMemoryInterface`). -Memory servers register themselves in the **Memory Name System (MNS)**, where they can be found by their (semantic) memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision"). -Memory servers receive data via **commits** (via the `WritingMemoryInterface`), and return data according to **queries** (via the `ReadingMemoryInterface`). - - -# Memory Levels - -A memory server stores data in a hierarchical structure consisting of several levels: -``` -- Grasping (Memory) - - Grasps (Core Segment) - - KnownObjectGraspPlanner (Provider Segment) - - my_object (Entity) - - t = 2021-03-09 14:24:20.064524 (Entity Snapshot) - - 00 (Entity Instance) - - 01 (Entity Instance) - - 02 (Entity Instance) -``` - -| Level | Key | Description | Examples | Implementation | -| ----- | --- | ----------- | -------- | -------------- | -| Memory | Name (String) | Semantic grouping of one or more core segments. Corresponds to one memory server. | Robot, Object, Vision | [Memory](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/Memory.h) | -| Core Segment | Name (String) | Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a _Core Segment Type_ which all provided data must comply to. | (Robot) Configuration, (Object) Classes, Known (Object) Instances, ImageRGB | [CoreSegment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/CoreSegment.h) | -| Provider Segment | Name (String) | Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a _Provider Segment Type_ which extends the _Core Segment Type_. | Primsense, SimTrack, MyGraspPlanner | [ProviderSegment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/ProviderSegment.h) | -| Entity | Name (String) | A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. | `image`, `some_object`, `some_object_grasps` | [Entity](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/Entity.h) | -| Entity Snapshot | Timestamp (`armem::Time` aka `IceUtil::Time`) | An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. | Image, object instance, grasp hypotheses at a time *t* | [EntitySnapshot](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/EntitySnapshot.h) | -| Entity Instance | Index (int) | One instance of the segment's ARON type. | left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) | [EntityInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/EntityInstance.h) | -| Entity Instance Metadata | -- | Metadata stored alongside the instance's data. | Further timesteps, confidence | [EntityInstanceMetadata](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem/core/EntityInstance.h) | - diff --git a/docs/armem/existing_memory_servers_and_segments/RobotState/Localization/res/frames.png b/etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/frames.png similarity index 100% rename from docs/armem/existing_memory_servers_and_segments/RobotState/Localization/res/frames.png rename to etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/frames.png diff --git a/docs/armem/existing_memory_servers_and_segments/RobotState/Localization/res/localization-armem.png b/etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/localization-armem.png similarity index 100% rename from docs/armem/existing_memory_servers_and_segments/RobotState/Localization/res/localization-armem.png rename to etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/localization-armem.png diff --git a/etc/doxygen/pages/armem/existing_memory_servers_and_segments/RobotState/localization.md b/etc/doxygen/pages/armem/existing_memory_servers_and_segments/RobotState/localization.md new file mode 100644 index 0000000000000000000000000000000000000000..5ccff189a7a92d15ed908fc7cc0537c2523d09b5 --- /dev/null +++ b/etc/doxygen/pages/armem/existing_memory_servers_and_segments/RobotState/localization.md @@ -0,0 +1,40 @@ +# Localization Segment (RobotState Memory) {#memory_system-localization} + +## Concept + +There exist various frames that are needed to obtain the robot's global pose (pose within in the global frame): + +| Name | Description | +|----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `global` | The "world" frame | +| `map` | During mapping, the robot does not have knowledge about the world but creates a local map (with `map` as root frame). All map features (e.g. point clouds, occupancy grids, ...) are linked to this frame. | +| `odom` | The odometry as an integration of the robot's velocity. | +| `robot` | The robot's root frame | + + + +For each transformation between frames, there exist individual components: + +| Involved Frames | Description | +|-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `global->map` | **Map registration:** given a scene (objects with known poses, e.g. walls, ...), this component obtains a static transformation between a the scene and the map. | +| `map->odom` | **Localization** The localization component computes the transformation `map->robot` and performs map corrections by updating `map->odom`. | +| `odom->robot` | **Odometry** | + + +### Memory design + +Within the core segment, each robot has its own provider segment (here: Armar6). Within the provider segment, entities describe the transformations between the frames ("Global,Map" -> [global->map]). + + + +### Using the localization memory + +See **RobotAPI/libraries/armem_robot_localization** + +There exist two classes: + +* armarx::armem::client::robot_state::localization::TransformWriter to send transformations to memory +* armarx::armem::client::robot_state::localization::TransformReader to obtain transformations between any of the frames; can also be used to obtain the "robot's global pose" + +For an example, see RobotComponents/components/carographer_mapping_and_localization (currently branch feature/cartographer_integration) diff --git a/etc/doxygen/pages/armem/existing_servers_segments.md b/etc/doxygen/pages/armem/existing_servers_segments.md new file mode 100644 index 0000000000000000000000000000000000000000..dc16b943bf1c622644ba3c5fe93f64a555564dda --- /dev/null +++ b/etc/doxygen/pages/armem/existing_servers_segments.md @@ -0,0 +1,89 @@ +# Existing Memory Servers and Segments {#memory_system-existing_servers_segments} + +This is a list of existing memory servers, their core segments and their \ref ARON "ARON" data types. + + +## RobotState {#memory_system-robot_state} + +* Memory Server: [`RobotStateMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/RobotStateMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------------------------------------------| +| Description | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Robot description (e.g. link to XML or URDF file) | Description of ARMAR-6 | +| Proprioception | [Proprioception](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_state/aron/Proprioception.xml) | Robot configuration and internal sensor values. | Proprioception of ARMAR-6 | +| \subpage memory_system-localization "Localization" | [Transform](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_localization/aron/Transform.xml) | Transformations between frames {world,map,robot} to retrieve global robot pose. | Transfrom from robot root frame to map frame | + +Missing: +- Robot calibration: should be part of `Description` core segment + + +## Object + +* Memory Server: [`ObjectMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ObjectMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------| +| Class | [ObjectClass](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectClass.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can` | +| Instance | [ObjectInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectInstance.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can/instance_1` | +| Attachments | [ObjectAttachment and ArticulatedObjectAttachment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/Attachment.xml) | | | + + +## Vision + +* Memory Server: [`ArMemVisionMemory` (VisionX)](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/tree/master/source/VisionX/components/armem/ArMemVisionMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------| +| ImageRGB | [`ImageRGB`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageRGB.xml) | RGB images (mono and stereo) of a camera | image | +| ImageDepth | [`ImageDepth`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageDepth.xml) | Depth images of an RGB-D or stereo camera | image | + + +## Speech + +* Memory Server: [`SpeechMemory` (SpeechX)](https://git.h2t.iar.kit.edu/sw/armarx/speechx/-/tree/master/source/SpeechX/components/SpeechMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-----------------------|-----------------|---------------------------------------------------------------------------|------------------| +| Command | `SpeechCommand` | Language commands from speech (topic `Speech_Commands`). | `command` | +| TextListenerInterface | `Text` | Data from `TextListenerInterface` topics. (`TextToSpeech`, `RobotAction`) | `text` | + +... + + +## Skills + +* Memory Server: [`SkillsMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/SkillsMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-------------------|--------------|-------------------------------------|------------------| +| Statechart | `Transition` | Transitions in ArmarX state charts. | tbd | + + +## Example + +An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development. + +* Memory Server: [`ExampleMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/ExampleMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------| +| ExampleData | [ExampleData](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ExampleMemory/aron/ExampleData.xml) | Some example data. | example | + + +## GeneralPurpose + +This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server. + +* Memory Server: [`GeneralPurposeMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/GeneralPurposeMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|----------------------------------------|------|-------------|------------------| +| None. You can add segments on the fly. | | | | + diff --git a/docs/armem/how_to_create_a_new_core_segment_or_memory_server/README.md b/etc/doxygen/pages/armem/how_to_create_a_new_core_segment_or_memory_server.md similarity index 99% rename from docs/armem/how_to_create_a_new_core_segment_or_memory_server/README.md rename to etc/doxygen/pages/armem/how_to_create_a_new_core_segment_or_memory_server.md index 9aaf1d9795a9ea38ae6414dc39ad03566f9f13da..c9a0cef826065e79f85e43e8929c64203b19a70d 100644 --- a/docs/armem/how_to_create_a_new_core_segment_or_memory_server/README.md +++ b/etc/doxygen/pages/armem/how_to_create_a_new_core_segment_or_memory_server.md @@ -1,3 +1,5 @@ +# How to Create a New Core segment or Memory Server {#memory_system-how_to_create_a_new_core_segment_or_memory_server} + In order to make a new type of data available in the memory system, you first need to create an ARON XML file. Please refer to [Aron/CodeGeneration](Aron/CodeGeneration) if you want to know how to do so. [[_TOC_]] @@ -390,4 +392,4 @@ When you now start your new memory server and the MemoryNameSystem (MNS) (e.g. u That's it! You now have a memory server providing a place where your nice and interesting data can feel welcome, and where interested listeners would look to find it. -[Next, we learn how to commit data to the memory, and query data from the memory.](ArMem/How-to-read-from-and-write-to-a-Memory-Server) \ No newline at end of file +[Next, we learn how to commit data to the memory, and query data from the memory.](ArMem/How-to-read-from-and-write-to-a-Memory-Server) diff --git a/etc/doxygen/pages/armem/introduction.md b/etc/doxygen/pages/armem/introduction.md new file mode 100644 index 0000000000000000000000000000000000000000..f96b6b1139744b649075c53b8d690ebb4a8f95b6 --- /dev/null +++ b/etc/doxygen/pages/armem/introduction.md @@ -0,0 +1,53 @@ +# Introduction to the Memory System {#memory_system-introduction} + +## Distributed Memory + +The ArMem memory system is distributed: It consists of several **memory servers** storing data in a hierarchical +structure. +A memory server can be defined in any ArmarX package and defines which kind of data it stores +(e.g. robot state, object classes and instances, visual data, speech, ...). +As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization, +aggregation, interpolation or prediction. +As each server is a separate component describing its own structure, it is simple to add new memory servers containing +special data types, which are defined in ArmarX packages further down in the dependency chain. + +All memory servers follow the same hierarchical data structure and self-describing data format. +The data structure consists of several levels, storing histories/timelines of different entities stored in different +segments (see "Memory Levels" below). +The common data format is \ref Aron, which is a self-describing, hierarchical data format, +allowing extensive introspection as well as general storage. +Each item in the memory (i.e. entries in all levels) can be identified with a **Memory ID**, which contains the keys +of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an +entity instance ID specifies the whole path to the leaf of the data structure. + +Technically, each memory server is one ArmarX component implementing the `armarx::armem::server::MemoryInterface` +(comprising a `armarx::armem::server::ReadingMemoryInterface` and a `armarx::armem::server::WritingMemoryInterface`). +Memory servers register themselves in the **Memory Name System (MNS)**, where they can be found by their (semantic) +memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision"). +Memory servers receive data via **commits** (via the `armarx::armem::server::WritingMemoryInterface`), +and return data according to **queries** (via the `armarx::armem::server::ReadingMemoryInterface`). + + +## Memory Levels + +A memory server stores data in a hierarchical structure consisting of several levels: +``` +- Grasping (Memory) + - Grasps (Core Segment) + - KnownObjectGraspPlanner (Provider Segment) + - my_object (Entity) + - t = 2021-03-09 14:24:20.064524 (Entity Snapshot) + - 00 (Entity Instance) + - 01 (Entity Instance) + - 02 (Entity Instance) +``` + +| Level | Key | Description | Examples | Implementation | +|--------------------------|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------| +| Memory | Name (String) | Semantic grouping of one or more core segments. Corresponds to one memory server. | Robot, Object, Vision | `armarx::armem::wm::Memory` | +| Core Segment | Name (String) | Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a _Core Segment Type_ which all provided data must comply to. | (Robot) Proprioception, (Object) Classes, Known (Object) Instances, ImageRGB | `armarx::armem::wm::CoreSegment` | +| Provider Segment | Name (String) | Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a _Provider Segment Type_ which extends the _Core Segment Type_. | Primesense, MyObjectLocalizer, MyGraspPlanner | `armarx::armem::wm::ProviderSegment` | +| Entity | Name (String) | A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. | `image`, `some_object`, `some_object_grasps` | `armarx::armem::wm::Entity` | +| Entity Snapshot | Timestamp (`armarx::armem::Time` aka `armarx::DateTime`) | An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. | Image, object instance, grasp hypotheses at a time *t* | `armarx::armem::wm::EntitySnapshot` | +| Entity Instance | Index (int) | One instance of the segment's ARON type. | left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) | `armarx::armem::wm::EntityInstance` | +| Entity Instance Metadata | -- | Metadata stored alongside the instance's data. | Further timesteps, confidence | `armarx::armem::wm::EntityInstanceMetadata` | diff --git a/etc/doxygen/pages/armem/memory_system.md b/etc/doxygen/pages/armem/memory_system.md new file mode 100644 index 0000000000000000000000000000000000000000..9abbee0639a076771fc2cfe49da7b45a42a941cd --- /dev/null +++ b/etc/doxygen/pages/armem/memory_system.md @@ -0,0 +1,8 @@ +# Memory System {#memory_system} + +- \subpage memory_system-introduction +- \subpage memory_system-existing_servers_segments +- \subpage memory_system-how_to_create_a_new_core_segment_or_memory_server + + +