diff --git a/etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/frames.png b/etc/doxygen/images/armem/RobotState/localization/frames.png similarity index 100% rename from etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/frames.png rename to etc/doxygen/images/armem/RobotState/localization/frames.png diff --git a/etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/localization-armem.png b/etc/doxygen/images/armem/RobotState/localization/localization-armem.png similarity index 100% rename from etc/doxygen/images/armem/existing_memory_servers_and_segments/RobotState/localization/localization-armem.png rename to etc/doxygen/images/armem/RobotState/localization/localization-armem.png diff --git a/etc/doxygen/pages/armem/existing_memory_servers_and_segments/RobotState/localization.md b/etc/doxygen/pages/armem/RobotState/localization.md similarity index 100% rename from etc/doxygen/pages/armem/existing_memory_servers_and_segments/RobotState/localization.md rename to etc/doxygen/pages/armem/RobotState/localization.md diff --git a/etc/doxygen/pages/armem/existing_servers_segments.md b/etc/doxygen/pages/armem/existing_servers_segments.md deleted file mode 100644 index dc16b943bf1c622644ba3c5fe93f64a555564dda..0000000000000000000000000000000000000000 --- a/etc/doxygen/pages/armem/existing_servers_segments.md +++ /dev/null @@ -1,89 +0,0 @@ -# Existing Memory Servers and Segments {#memory_system-existing_servers_segments} - -This is a list of existing memory servers, their core segments and their \ref ARON "ARON" data types. - - -## RobotState {#memory_system-robot_state} - -* Memory Server: [`RobotStateMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/RobotStateMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------------------------------------------| -| Description | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Robot description (e.g. link to XML or URDF file) | Description of ARMAR-6 | -| Proprioception | [Proprioception](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_state/aron/Proprioception.xml) | Robot configuration and internal sensor values. | Proprioception of ARMAR-6 | -| \subpage memory_system-localization "Localization" | [Transform](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_localization/aron/Transform.xml) | Transformations between frames {world,map,robot} to retrieve global robot pose. | Transfrom from robot root frame to map frame | - -Missing: -- Robot calibration: should be part of `Description` core segment - - -## Object - -* Memory Server: [`ObjectMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ObjectMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------| -| Class | [ObjectClass](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectClass.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can` | -| Instance | [ObjectInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectInstance.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can/instance_1` | -| Attachments | [ObjectAttachment and ArticulatedObjectAttachment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/Attachment.xml) | | | - - -## Vision - -* Memory Server: [`ArMemVisionMemory` (VisionX)](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/tree/master/source/VisionX/components/armem/ArMemVisionMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------| -| ImageRGB | [`ImageRGB`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageRGB.xml) | RGB images (mono and stereo) of a camera | image | -| ImageDepth | [`ImageDepth`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageDepth.xml) | Depth images of an RGB-D or stereo camera | image | - - -## Speech - -* Memory Server: [`SpeechMemory` (SpeechX)](https://git.h2t.iar.kit.edu/sw/armarx/speechx/-/tree/master/source/SpeechX/components/SpeechMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-----------------------|-----------------|---------------------------------------------------------------------------|------------------| -| Command | `SpeechCommand` | Language commands from speech (topic `Speech_Commands`). | `command` | -| TextListenerInterface | `Text` | Data from `TextListenerInterface` topics. (`TextToSpeech`, `RobotAction`) | `text` | - -... - - -## Skills - -* Memory Server: [`SkillsMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/SkillsMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------|--------------|-------------------------------------|------------------| -| Statechart | `Transition` | Transitions in ArmarX state charts. | tbd | - - -## Example - -An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development. - -* Memory Server: [`ExampleMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/ExampleMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------| -| ExampleData | [ExampleData](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ExampleMemory/aron/ExampleData.xml) | Some example data. | example | - - -## GeneralPurpose - -This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server. - -* Memory Server: [`GeneralPurposeMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/GeneralPurposeMemory) -* Core Segments: - -| Core Segment Name | Type | Description | Example Entities | -|----------------------------------------|------|-------------|------------------| -| None. You can add segments on the fly. | | | | - diff --git a/etc/doxygen/pages/armem/introduction.md b/etc/doxygen/pages/armem/introduction.md deleted file mode 100644 index f96b6b1139744b649075c53b8d690ebb4a8f95b6..0000000000000000000000000000000000000000 --- a/etc/doxygen/pages/armem/introduction.md +++ /dev/null @@ -1,53 +0,0 @@ -# Introduction to the Memory System {#memory_system-introduction} - -## Distributed Memory - -The ArMem memory system is distributed: It consists of several **memory servers** storing data in a hierarchical -structure. -A memory server can be defined in any ArmarX package and defines which kind of data it stores -(e.g. robot state, object classes and instances, visual data, speech, ...). -As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization, -aggregation, interpolation or prediction. -As each server is a separate component describing its own structure, it is simple to add new memory servers containing -special data types, which are defined in ArmarX packages further down in the dependency chain. - -All memory servers follow the same hierarchical data structure and self-describing data format. -The data structure consists of several levels, storing histories/timelines of different entities stored in different -segments (see "Memory Levels" below). -The common data format is \ref Aron, which is a self-describing, hierarchical data format, -allowing extensive introspection as well as general storage. -Each item in the memory (i.e. entries in all levels) can be identified with a **Memory ID**, which contains the keys -of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an -entity instance ID specifies the whole path to the leaf of the data structure. - -Technically, each memory server is one ArmarX component implementing the `armarx::armem::server::MemoryInterface` -(comprising a `armarx::armem::server::ReadingMemoryInterface` and a `armarx::armem::server::WritingMemoryInterface`). -Memory servers register themselves in the **Memory Name System (MNS)**, where they can be found by their (semantic) -memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision"). -Memory servers receive data via **commits** (via the `armarx::armem::server::WritingMemoryInterface`), -and return data according to **queries** (via the `armarx::armem::server::ReadingMemoryInterface`). - - -## Memory Levels - -A memory server stores data in a hierarchical structure consisting of several levels: -``` -- Grasping (Memory) - - Grasps (Core Segment) - - KnownObjectGraspPlanner (Provider Segment) - - my_object (Entity) - - t = 2021-03-09 14:24:20.064524 (Entity Snapshot) - - 00 (Entity Instance) - - 01 (Entity Instance) - - 02 (Entity Instance) -``` - -| Level | Key | Description | Examples | Implementation | -|--------------------------|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------| -| Memory | Name (String) | Semantic grouping of one or more core segments. Corresponds to one memory server. | Robot, Object, Vision | `armarx::armem::wm::Memory` | -| Core Segment | Name (String) | Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a _Core Segment Type_ which all provided data must comply to. | (Robot) Proprioception, (Object) Classes, Known (Object) Instances, ImageRGB | `armarx::armem::wm::CoreSegment` | -| Provider Segment | Name (String) | Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a _Provider Segment Type_ which extends the _Core Segment Type_. | Primesense, MyObjectLocalizer, MyGraspPlanner | `armarx::armem::wm::ProviderSegment` | -| Entity | Name (String) | A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. | `image`, `some_object`, `some_object_grasps` | `armarx::armem::wm::Entity` | -| Entity Snapshot | Timestamp (`armarx::armem::Time` aka `armarx::DateTime`) | An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. | Image, object instance, grasp hypotheses at a time *t* | `armarx::armem::wm::EntitySnapshot` | -| Entity Instance | Index (int) | One instance of the segment's ARON type. | left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) | `armarx::armem::wm::EntityInstance` | -| Entity Instance Metadata | -- | Metadata stored alongside the instance's data. | Further timesteps, confidence | `armarx::armem::wm::EntityInstanceMetadata` | diff --git a/etc/doxygen/pages/armem/memory_system.md b/etc/doxygen/pages/armem/memory_system.md index 72da8b5187cf0dad4c2dab207f93f773c1c29c44..a275f5f4c351a632c2d30f00e82fe855ab8d75d4 100644 --- a/etc/doxygen/pages/armem/memory_system.md +++ b/etc/doxygen/pages/armem/memory_system.md @@ -1,5 +1,149 @@ # Memory System {#memory_system} -- \subpage memory_system-introduction -- \subpage memory_system-existing_servers_segments +\tableofcontents + - \subpage memory_system-how_to_create_a_new_core_segment_or_memory_server + +## Introduction to the Memory System {#memory_system-introduction} + +### Distributed Memory + +The ArMem memory system is distributed: It consists of several **memory servers** storing data in a hierarchical +structure. +A memory server can be defined in any ArmarX package and defines which kind of data it stores +(e.g. robot state, object classes and instances, visual data, speech, ...). +As a memory server knows which data it stores, it can provide special behaviour, such as custom visualization, +aggregation, interpolation or prediction. +As each server is a separate component describing its own structure, it is simple to add new memory servers containing +special data types, which are defined in ArmarX packages further down in the dependency chain. + +All memory servers follow the same hierarchical data structure and self-describing data format. +The data structure consists of several levels, storing histories/timelines of different entities stored in different +segments (see "Memory Levels" below). +The common data format is \ref Aron, which is a self-describing, hierarchical data format, +allowing extensive introspection as well as general storage. +Each item in the memory (i.e. entries in all levels) can be identified with a **Memory ID**, which contains the keys +of each (specified) level. For example, a core segment ID specifies the memory name and core segment name, while an +entity instance ID specifies the whole path to the leaf of the data structure. + +Technically, each memory server is one ArmarX component implementing the `armarx::armem::server::MemoryInterface` +(comprising a `armarx::armem::server::ReadingMemoryInterface` and a `armarx::armem::server::WritingMemoryInterface`). +Memory servers register themselves in the **Memory Name System (MNS)**, where they can be found by their (semantic) +memory name (which is usually a short, human-readable name such as "Robot", "Object" or "Vision"). +Memory servers receive data via **commits** (via the `armarx::armem::server::WritingMemoryInterface`), +and return data according to **queries** (via the `armarx::armem::server::ReadingMemoryInterface`). + + +### Memory Levels + +A memory server stores data in a hierarchical structure consisting of several levels: +``` +- Grasping (Memory) + - Grasps (Core Segment) + - KnownObjectGraspPlanner (Provider Segment) + - my_object (Entity) + - t = 2021-03-09 14:24:20.064524 (Entity Snapshot) + - 00 (Entity Instance) + - 01 (Entity Instance) + - 02 (Entity Instance) +``` + +| Level | Key | Description | Examples | Implementation | +|--------------------------|----------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|---------------------------------------------| +| Memory | Name (String) | Semantic grouping of one or more core segments. Corresponds to one memory server. | Robot, Object, Vision | `armarx::armem::wm::Memory` | +| Core Segment | Name (String) | Building block of a memory containing homogeneous data of a specific (ARON) type. Specifies a _Core Segment Type_ which all provided data must comply to. | (Robot) Proprioception, (Object) Classes, Known (Object) Instances, ImageRGB | `armarx::armem::wm::CoreSegment` | +| Provider Segment | Name (String) | Sub-segment of a core segment which contains the data from a single source (e.g. a camera, a method, a component). Can define a _Provider Segment Type_ which extends the _Core Segment Type_. | Primesense, MyObjectLocalizer, MyGraspPlanner | `armarx::armem::wm::ProviderSegment` | +| Entity | Name (String) | A single thing or concept whose properties change over time. An entity has a history / timeline, i.e. a sequence of snapshots storing the entity's properties at a specific point in time. | `image`, `some_object`, `some_object_grasps` | `armarx::armem::wm::Entity` | +| Entity Snapshot | Timestamp (`armarx::armem::Time` aka `armarx::DateTime`) | An entry of an entity's history, i.e. the state of an entity at a specific point in time. Can contain multiple (entitiy) instances. | Image, object instance, grasp hypotheses at a time *t* | `armarx::armem::wm::EntitySnapshot` | +| Entity Instance | Index (int) | One instance of the segment's ARON type. | left/right stereo image (at time t), object (at time t), grasp hypothesis (one of many at time t) | `armarx::armem::wm::EntityInstance` | +| Entity Instance Metadata | -- | Metadata stored alongside the instance's data. | Further timesteps, confidence | `armarx::armem::wm::EntityInstanceMetadata` | + + +## Existing Memory Servers and Segments {#memory_system-existing_servers_segments} + +This is a list of existing memory servers, their core segments and their \ref ARON "ARON" data types. + + +### RobotState {#memory_system-robot_state} + +* Memory Server: [`RobotStateMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/RobotStateMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|----------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------|----------------------------------------------| +| Description | [RobotDescription](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot/aron/RobotDescription.xml) | Robot description (e.g. link to XML or URDF file) | Description of ARMAR-6 | +| Proprioception | [Proprioception](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_state/aron/Proprioception.xml) | Robot configuration and internal sensor values. | Proprioception of ARMAR-6 | +| \subpage memory_system-localization "Localization" | [Transform](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_robot_localization/aron/Transform.xml) | Transformations between frames {world,map,robot} to retrieve global robot pose. | Transfrom from robot root frame to map frame | + +Missing: +- Robot calibration: should be part of `Description` core segment + + +### Object + +* Memory Server: [`ObjectMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ObjectMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|--------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|-----------------------------------------------------| +| Class | [ObjectClass](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectClass.xml) | Static description of a known object class. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can` | +| Instance | [ObjectInstance](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/ObjectInstance.xml) | Localized object instance. Entity names are (ArmarXObjects) object IDs. | `KIT/CoffeeFilters`, `YCB/001_chips_can/instance_1` | +| Attachments | [ObjectAttachment and ArticulatedObjectAttachment](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/libraries/armem_objects/aron/Attachment.xml) | | | + + +### Vision + +* Memory Server: [`ArMemVisionMemory` (VisionX)](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/tree/master/source/VisionX/components/armem/ArMemVisionMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-------------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|------------------| +| ImageRGB | [`ImageRGB`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageRGB.xml) | RGB images (mono and stereo) of a camera | image | +| ImageDepth | [`ImageDepth`](https://git.h2t.iar.kit.edu/sw/armarx/visionx/-/blob/master/source/VisionX/libraries/ArMem/aron/ImageDepth.xml) | Depth images of an RGB-D or stereo camera | image | + + +### Speech + +* Memory Server: [`SpeechMemory` (SpeechX)](https://git.h2t.iar.kit.edu/sw/armarx/speechx/-/tree/master/source/SpeechX/components/SpeechMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-----------------------|-----------------|---------------------------------------------------------------------------|------------------| +| Command | `SpeechCommand` | Language commands from speech (topic `Speech_Commands`). | `command` | +| TextListenerInterface | `Text` | Data from `TextListenerInterface` topics. (`TextToSpeech`, `RobotAction`) | `text` | + +... + + +### Skills + +* Memory Server: [`SkillsMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/SkillsMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-------------------|--------------|-------------------------------------|------------------| +| Statechart | `Transition` | Transitions in ArmarX state charts. | tbd | + + +### Example + +An example memory server (alongside a matching client) showing how to implement a standard memory server and test the framework during development. + +* Memory Server: [`ExampleMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/ExampleMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|-------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------|------------------| +| ExampleData | [ExampleData](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/blob/master/source/RobotAPI/components/armem/server/ExampleMemory/aron/ExampleData.xml) | Some example data. | example | + + +### GeneralPurpose + +This memory is meant to allow for a quick-and-dirty integration of your data type in the memory framework. When your development finishes a stable state, consider creating moving the core segment(s) to an existing memory server or creating a new dedicated memory server. + +* Memory Server: [`GeneralPurposeMemory` (RobotAPI)](https://git.h2t.iar.kit.edu/sw/armarx/robot-api/-/tree/master/source/RobotAPI/components/armem/server/GeneralPurposeMemory) +* Core Segments: + +| Core Segment Name | Type | Description | Example Entities | +|----------------------------------------|------|-------------|------------------| +| None. You can add segments on the fly. | | | |