[Хостинг, Децентрализованные сети, Хранилища данных, Распределённые системы] Decentralized Torrent storage in DHT
Автор
Сообщение
news_bot ®
Стаж: 6 лет 3 месяца
Сообщений: 27286
The DHT system has existed for many years now, and torrents along with it, which we successfully use to get any information we want.Together with this system, there are commands to interact with it. There are not many of them, but only two are needed to create a decentralized database: put and get. This is what will be discussed below...Logically, everyone can understand. Put is to put. And Get is to get. So you can reliably put up to 1000 bytes of any information with the command Put. And reliably this information will be stored in DHT for about one hour. Get is to get what you put in. It's simple.The Put command has two types. The first is put without the possibility of change. The second is put with the ability to change. This means that you can put 1000 bytes into DHT and change them as you want.To put or change the mutable data needs 2 ed25519 keys. A public key and a private key. And everyone who has these two keys can change the data as he wants.remind how private and public keys workInformation can be encrypted with a private key and decrypted with a public key. That's why these keys are paired. It is impossible to guess the private key with the public key.On this it is possible to build a decentralized databaseThere are a lot of variants. It's a matter of your imagination, but you can use, for example, this one.All users = peers have a public and a private key.User 1 wants to change the database. He gets the contents of the DHT using Get and the public key. In the received data, it could be anything. For us it will be the sha1 link of the torrent from which the torrent can be downloaded. It will be about 20 bytes in size. Using this link, it downloads the torrent in which the database is located. Modifies the database. Generates a torrent (obtaining a sha1 key) and distributes it. Publishes with Put obtained sha1 key which can be used to download the new, modified DB.User 2 wants to change DB....analogically...In order to keep data in DHT, it is necessary to synchronize peers on a regular interval. Synchronization is a Get with public key via udp. It's very inexpensive. And if the data has changed then download it. This is a little more expensive, but here, too, you can put a limit on the download speed, for example.If the data in the DHT is gone, the peer that wants to sync just does a Put with the last data it has, or with clean data.Now a little bit about why this is a database and not a file exchangeIn a 1000 bytes you can save anything you want. And there is an infinite amount of information in the transmitted database. Since the user does not know the public and private key, he can not publish something himself. Only the program can do that. And the program, depending on the data in those 1000 bytes, can do anything. It can get the user's lock and set it to read-only mode. It turns out it is a database with access rights to change certain information. Especially, it will be similar to the database, if the downloaded data are encrypted and stored in a single file without extension. Of course, there may be some problems here, but they are all solvable.Now about the disadvantages and problemsFor example, if user 1 published a new hash of a modified database, but didn't distribute it. In DHT you can store 5 sha1 records, this is 100+ bytes in total and if any user could not download the changed version, he downloads the next of 5 and then republishes the one that is downloaded. Correspondingly, other users will not try to download what is not downloaded and get the DB faster. If there are more users, the better and faster the database will work.The main problem with such a system is the waiting time. Publication (Put) takes from 20 to 60 seconds + time for someone to download this data. If the user does not shut down the program, it is only 20-60 seconds. But receiving data as it is torrents - is done at maximal speed of device with use of many torrent technologies and protocols. Downloading only modified data? I think it is possible, but don't ask how.The second important problem is that every user has a program in which the public key and the private key are stored. And if the program is hacked, it is possible to get these keys and use them to publish false data. Here it is possible to make up a lot of ways to deal with it. Perhaps in the comments will suggest some more. But for example: it is possible to store a key in encrypted form and decrypt it only when using it. It is possible to encrypt the database with a separate key, so that an attacker needs to calculate this key too. It is possible to store keys in DHT too, and change them periodically or on site (this is unhackable). There are as many ways as your imagination is rich. Therefore, it is impossible to say for sure that this is a security vulnerability.Technically it is possible to do this based on any torrent library. For example Libtorrent. It weighs only 2.5Mb after compilation, it is written in C++ and works as fast as possible. There is some technical information about Put. A similar system is used in my "Media Library" app, for publishing playlists. I already have even an admin interface for moderation. Everything works successfully. Enjoy it!Because the karma system on the site does not work properly, I can't comment on my own articles. Therefore, you may consider that comments on the article are disabled. Write your questions in personal messages. There, perhaps, I will answer.
===========
Источник:
habr.com
===========
Похожие новости:
- [Информационная безопасность, Хранение данных, Хранилища данных, Data Engineering] Как токенизация упрощает работу с чувствительными данными
- [IT-инфраструктура, Хранение данных, Хранилища данных, Инженерные системы] Проверка двигателя на прочность: как мы тестируем динамические ИБП
- [Администрирование баз данных, Распределённые системы, Микросервисы] Как e2e автотесты на Selenide помогают QA-команде при частых релизах
- [Хостинг, Децентрализованные сети, Хранилища данных, Распределённые системы] Децентрализованное Torrent хранилище в DHT
- [Хостинг, Разработка веб-сайтов, Разработка под Android, API, Транспорт] Как реализовать отслеживание местоположения андроид устройства на своем сайте
- [IT-инфраструктура, Хранение данных, Хранилища данных, Облачные сервисы] Как выбрать облачную систему хранения данных, чтобы получить лучшую производительность и оптимизировать стоимость
- [Хостинг, Хранение данных, Законодательство в IT, Облачные сервисы] Лицензии связи
- [Децентрализованные сети, Сетевые технологии] АНОНС: Систематизированный обзор литературы по военным SDN сетям (перевод) (перевод)
- [Микроформаты, Децентрализованные сети, Расширения для браузеров, Управление медиа] Смотрим любое кино мгновенно
- [Анализ и проектирование систем, Промышленное программирование, Распределённые системы] Устройство гетерогенного кластера выполнения задач. Доклад Яндекса
Теги для поиска: #_hosting (Хостинг), #_detsentralizovannye_seti (Децентрализованные сети), #_hranilischa_dannyh (Хранилища данных), #_raspredelennye_sistemy (Распределённые системы), #_dht, #_torrents, #_decentralization, #_decentralised_networks, #_hosting (
Хостинг
), #_detsentralizovannye_seti (
Децентрализованные сети
), #_hranilischa_dannyh (
Хранилища данных
), #_raspredelennye_sistemy (
Распределённые системы
)
Вы не можете начинать темы
Вы не можете отвечать на сообщения
Вы не можете редактировать свои сообщения
Вы не можете удалять свои сообщения
Вы не можете голосовать в опросах
Вы не можете прикреплять файлы к сообщениям
Вы не можете скачивать файлы
Текущее время: 12-Май 04:49
Часовой пояс: UTC + 5
| Автор | Сообщение |
|---|---|
|
news_bot ®
Стаж: 6 лет 3 месяца |
|
 The DHT system has existed for many years now, and torrents along with it, which we successfully use to get any information we want.Together with this system, there are commands to interact with it. There are not many of them, but only two are needed to create a decentralized database: put and get. This is what will be discussed below...Logically, everyone can understand. Put is to put. And Get is to get. So you can reliably put up to 1000 bytes of any information with the command Put. And reliably this information will be stored in DHT for about one hour. Get is to get what you put in. It's simple.The Put command has two types. The first is put without the possibility of change. The second is put with the ability to change. This means that you can put 1000 bytes into DHT and change them as you want.To put or change the mutable data needs 2 ed25519 keys. A public key and a private key. And everyone who has these two keys can change the data as he wants.remind how private and public keys workInformation can be encrypted with a private key and decrypted with a public key. That's why these keys are paired. It is impossible to guess the private key with the public key.On this it is possible to build a decentralized databaseThere are a lot of variants. It's a matter of your imagination, but you can use, for example, this one.All users = peers have a public and a private key.User 1 wants to change the database. He gets the contents of the DHT using Get and the public key. In the received data, it could be anything. For us it will be the sha1 link of the torrent from which the torrent can be downloaded. It will be about 20 bytes in size. Using this link, it downloads the torrent in which the database is located. Modifies the database. Generates a torrent (obtaining a sha1 key) and distributes it. Publishes with Put obtained sha1 key which can be used to download the new, modified DB.User 2 wants to change DB....analogically...In order to keep data in DHT, it is necessary to synchronize peers on a regular interval. Synchronization is a Get with public key via udp. It's very inexpensive. And if the data has changed then download it. This is a little more expensive, but here, too, you can put a limit on the download speed, for example.If the data in the DHT is gone, the peer that wants to sync just does a Put with the last data it has, or with clean data.Now a little bit about why this is a database and not a file exchangeIn a 1000 bytes you can save anything you want. And there is an infinite amount of information in the transmitted database. Since the user does not know the public and private key, he can not publish something himself. Only the program can do that. And the program, depending on the data in those 1000 bytes, can do anything. It can get the user's lock and set it to read-only mode. It turns out it is a database with access rights to change certain information. Especially, it will be similar to the database, if the downloaded data are encrypted and stored in a single file without extension. Of course, there may be some problems here, but they are all solvable.Now about the disadvantages and problemsFor example, if user 1 published a new hash of a modified database, but didn't distribute it. In DHT you can store 5 sha1 records, this is 100+ bytes in total and if any user could not download the changed version, he downloads the next of 5 and then republishes the one that is downloaded. Correspondingly, other users will not try to download what is not downloaded and get the DB faster. If there are more users, the better and faster the database will work.The main problem with such a system is the waiting time. Publication (Put) takes from 20 to 60 seconds + time for someone to download this data. If the user does not shut down the program, it is only 20-60 seconds. But receiving data as it is torrents - is done at maximal speed of device with use of many torrent technologies and protocols. Downloading only modified data? I think it is possible, but don't ask how.The second important problem is that every user has a program in which the public key and the private key are stored. And if the program is hacked, it is possible to get these keys and use them to publish false data. Here it is possible to make up a lot of ways to deal with it. Perhaps in the comments will suggest some more. But for example: it is possible to store a key in encrypted form and decrypt it only when using it. It is possible to encrypt the database with a separate key, so that an attacker needs to calculate this key too. It is possible to store keys in DHT too, and change them periodically or on site (this is unhackable). There are as many ways as your imagination is rich. Therefore, it is impossible to say for sure that this is a security vulnerability.Technically it is possible to do this based on any torrent library. For example Libtorrent. It weighs only 2.5Mb after compilation, it is written in C++ and works as fast as possible. There is some technical information about Put. A similar system is used in my "Media Library" app, for publishing playlists. I already have even an admin interface for moderation. Everything works successfully. Enjoy it!Because the karma system on the site does not work properly, I can't comment on my own articles. Therefore, you may consider that comments on the article are disabled. Write your questions in personal messages. There, perhaps, I will answer. =========== Источник: habr.com =========== Похожие новости:
Хостинг ), #_detsentralizovannye_seti ( Децентрализованные сети ), #_hranilischa_dannyh ( Хранилища данных ), #_raspredelennye_sistemy ( Распределённые системы ) |
|
Вы не можете начинать темы
Вы не можете отвечать на сообщения
Вы не можете редактировать свои сообщения
Вы не можете удалять свои сообщения
Вы не можете голосовать в опросах
Вы не можете прикреплять файлы к сообщениям
Вы не можете скачивать файлы
Вы не можете отвечать на сообщения
Вы не можете редактировать свои сообщения
Вы не можете удалять свои сообщения
Вы не можете голосовать в опросах
Вы не можете прикреплять файлы к сообщениям
Вы не можете скачивать файлы
Текущее время: 12-Май 04:49
Часовой пояс: UTC + 5