Xiangdong Huang created COMDEV-403:
--------------------------------------
Summary: Apache IoTDB: Metadata (Schema) Storage Engine
Key: COMDEV-403
URL: https://issues.apache.org/jira/browse/COMDEV-403
Project: Community Development
Issue Type: Task
Components: GSoC/Mentoring ideas
Reporter: Xiangdong Huang
Apache IoTDB [1] is an Open Source IoT database designed to meet the rigorous
data, storage, and analytics requirements of large-scale Internet of Things
(IoT) and Industrial Internet of Things (IIoT) applications.
Different with traditional relational databases, IoTDB uses tree-based
structure in memory to manage the schema (a.k.a, metadata), and use a
Write-Ahead-Log-like file structure to persist the schema.
Now, each time series will take about 300 Bytes in memory. However, an IoTDB
instance may manage more than 100 million time series, which may take more than
30GB memory.
Therefore, we'd like to re-design the schema management module.
1. File: Persist the tree on disk like a b-tree.
2. WAL: implement the WAL of the metadata. So we can update the tree on disk in
batch, rather than one operation by one.
3. Cache: we may have no memory to load the whole tree. So a cache is needed
and query from the tree on disk is needed.
What knowledge you need to know:
1. Java
2. Basic design idea about Database [2]
[1] https://iotdb.apache.org
[2]
http://pages.cs.wisc.edu/~dbbook/openAccess/firstEdition/slides/pdfslides/mod2l1.pdf
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
