/t/ - Technology

Discussion of Technology

Index Catalog Archive Bottom Refresh
Options
Subject
Message

Max message length: 12000

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)

Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

US Election Thread

8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

You may also be interested in: AI

(1.09 MB OrbitDB.pdf)

(454.35 KB 1920x1912 shit-the-fed-pot.jpg)

DWeb board Anonymous 03/04/2022 (Fri) 20:17:53 No. 7678
How hard would it be to make a functioning imageboard (or textboard) using IPFS and OrbitDB/GUN, or Hypercore/Dat? It would be immune to deplatforming by hosting providers (and DNS providers depending on how it's addressed). https://ipfs.io/ https://orbitdb.org/ https://github.com/amark/gun / https://gun.eco/ https://hypercore-protocol.org/ (formally https://www.datprotocol.com/) There's two chans on zeronet but you have to make an ID to use them
>>8972 I wasn't shilling any other existing site. I guess the mod just jumped the gun. MODS = FAGS as they say.
>>8973 Oh calm your tits, not like you're banned or nothing. Just make it again and I won't kill it.
>>8974 Did you ever read the book "The Diamond Age"? It's sci-fi where the economy is centered on this technology called "The Feed" which is essentially molecular assemblers that require a very expensive filtration system to prepare the raw atomic ingredients. Those in control of the purifiers also control what gets 3D printed, because each printer-assembler by law has software that only allows certain things to be manufactured and documents everything printed. On the periphery of the society is a group of people working to develop "The Seed" which is programmable seeds that you can place into the soil and they will bootstrap themselves and grow into whatever specified. Unregulated, and essentially uncontrollable. There is tension over the development of The Seed with people trying to develop it in secret (in the series it is people in a nanobot-induced trance that get spirited away into this underwater hive where they are in a mass orgy of collective computation to try to puzzle out the details of the technology), and the controllers of The Feed trying to suppress its development. Very interesting story, do recommend if you get the chance.
>>8975 yes I read the sequel to Snow Crash. It's allright. I still prefer Zodiac though.
>>8975 >"The Seed" >not "The Sneed" ngmi
IPFS is genuinely good.
>>8999 Yeah. The only issue I can see is that people tend to use ipfs gateways which are a point where censorship can occur. It should be designed so that each user automatically runs their own ipfs node in the browser as they browse the site and avoid gateways entirely.
>>8999 By the way OrbitDB runs on top of IPFS >>8970 Actually checking for "invalid" posts might not be an issue because I think post validity can be defined at the database level. In other words database writes into a thread could only occur. If posts are fully modular though, and Thread databses simply contain references to individual Posts database IPFS addresses rather than containing the post contents itself, the filtering becomes more of an issue. "Chans" -> "Boards" -> "Threads" -> "Posts" Ideally these would be sort of a hierarchically arranged fractal with interchangeable parts referencing each other, that can be swapped out and added or removed freely, but then making things conform to a specific standard also becomes a bit trickier.
>>9003 The opposite of the interchangeable model would be to have a single "site" simply be a database list of Posts, each post containing data on which thread or board it is in. Each peer would then seed the entire site database containing all posts, and all the arranging of posts into specific threads would occur on the frontend. This seems to me to be perhaps simpler, but inefficient. And interesting concept with the first model is that a "site" doesn't even have to have a single canonical list of boards. People could decide to 'subscribe' to different sets of boards from the same UI interface.
Thumbnailing is also nontrivial. How do you generate thumbnails and ensure they correspond to the full image? It seems there would have to be some sort of agreed-upon algorithm used to generate the thumbnail.
>>7678 Not that hard
>>9131 I was thinking maybe a "single thread board" like >>>/digi/ would be a good place to start.
Well since I didn't know enough to do this I'm making an imageboard in python/flask. Hopefully will help me understand how it works enough to then see what parts can be made p2p easier than others. A server would likely be two parts in the intermediate stage of not yet full-p2p An "indexer" (in this case using Python) that would handle things like: >Proper formatting and validation of posts >Hosting the database contains information on: >-what boards canonically exist >-what threads a given board canonically contains >-canonical thread order based on bumps >-what posts a given thread canonically contains. >-Moderation and access concerns. >-thumbnail concerns >>9005 Eventually these could be one-by-one shunted off onto OrbitDB or Gun or something until nothing remains. And the p2p side, which serves as a "seedbox" for all the actual content, which would gradually expand to include the aforementioned database elements. Most recently I've gotten bumping to work, lol.
Two additional issues: Most WebRTC/IPFS stuff seems to use javascript so people might not like having to enable scripts to be able to post and browse. A way to mask one's ip from others in the swarm is important to prevent everyone IP doxxing each other.
Something I've realized doing this: It could be possible to do it differently, but ideally each "parent" {eg. board} would have references to its children {eg. thread}, but not the other way around. So a board database object contains all the information about which threads it contains, but threads objects don't require any information about which board they are part of. Everything just references it's children via hashes/url. On the flip side if you did decide to have a thread contain information about it's parent board, you could then use the thread to go up the tree and find other threads in the same board. Interesting but not fundamentally necessarily for a basic implementation.
(52.96 KB 600x600 square-orbitdb.png)

I started working on something like this. Instead of doing it from the ground up I decided to fork lynxchan and convert everything over to orbitdb. If anyone wants to help feel free. https://gitgud.io/threshold862543/orbitchan
>>9494 Oops I meant jschan.
>>9005 I would say the uploader should generate the thumbnail if it exists at all. And then it gets stored in the database alongside the image. Since there is no server to do it, and having each viewer download the full image in order to generate a thumbnail is pointless.
>>9532 Yeah that sounds about right. The uploader could easily make fake thumbnails, but I guess that's not that big of a deal.
>>9533 What if the uploader spams the board with massive thumbnails and causes it to slow down for everyone? >>8970 >some form of consensus or standardized format >something like an agreed-upon standardized script This would have to be client-side, so what's stopping it from being modified to be malicious? It might be worth looking to see how other P2P applications solve this. Like Berty (P2P Signal) or the abovementioned Aether (>>8206).
>>9534 Spam can be helped by PoW but what if they post one huge thumbnail? There should be a way of limiting the size.
>>9535 I think what's going to develop is a sort of standard client-side configuration where you set what filesizes you want to load by default. I think in IPFS for example you can get data about the file type and size before you actually load the file. In a highly distributed system I think a lot of the parsing and arranging of data is going to be handled on the client side and "enforced" by convention, say a standard set of userscripts people generally use to browse the data and post. Part of the reason I'm interesting in doing this >>9494 is because by adapting a traditional server to a distributed system we can sort of work through what is easier than what, and what remains to be figured out.
>>9535 >>9535 Also something to consider is, in a distributed system, if you want to spam, you have to be the one seeding your spam. If you want to force everyone to download your 1 GB jpg, you have to seed it to them, so there's a bit of balance in that regard.
>>9534 Interesting. A lot of p2p messaging applications seem to be developing nowadays.
>>9537 True. Unless it's being relayed by other peers a la BitTorrent/IPFS. But if it's being relayed then the peers could probably enforce some kind of limit.
>>9543 In your chan-client settings, you could set something like: >max filesize to autodownload: 4 mb >filetypes to autodownload: jpg, png, gif Beyond that, it would try to load the corresponding thumbnail instead or just put the filename and extension as a link if it's something else like a .zip
>>9545 If nobody browsing (seeding) the thread has their setting above 10 mb, someone trying to post a 100 mb file would end up being the only seeder and have no downloaders.
I've been making a bit of progress on this >>9494. I moved the posts database to OrbitDB and fragmented it up so that replies are stored on a per-thread basis, but still need to implement a bunch of the functions to have it work properly. The benefit of fragmenting the database is avoiding have to keep a single massive oplog, and also for modularity; ideally a user could simply enter the hash-address of a thread and load it in full without going through a central server.
Found something cool that could be useful for this: https://github.com/freenet/locutus
(261.85 KB 1886x1883 twocatalogs.png)

Made a bit of progress on the overall design concept as well as implementation on the orbitchan project. Here you can see two catalog pages. The left is the traditional view served by the central server as you would see accessing an orbitchan (or vanilla jschan) catalog page. On the right we have a page generated and served by the local peer node, that gets all of its data over IPFS. There's some missing elements (including CSS) but I think highlights what's going on to an extent. Also released the basic foundation of the peer node server if anyone wants to mess with it: https://gitgud.io/threshold862543/orbitchan-peer
(171.92 KB 904x941 twocatalogscss2.png)

Got css working on the peer node side.
(96.63 KB 887x1641 thread.png)

>>9820 And the basis of a thread page. The next big thing to do is probably to get the files uploading and downloading to and from IPFS properly.
(159.82 KB 1789x845 sidebyside.png)

>>9828 Got images to load over IPFS. Here's how the two catalogs compare now. The difference in background color is just because I scrolled down to get things to line up. Right is obtained over IPFS (except for the board name and description which are obtained over HTTP for now) and the left is the traditionally served HTML page from the central server. As I'm doing this a lot of the quality-of-life stuff that should be handled is becoming more clear.
>>9903 Making a bit of progress on this, mainly fixing up a lot of the functionality that broke porting the database. The most recent thing I did was having the central-servers "classic view" also get post files via an proxy to the ipfs gateway on the backend, so that the static file delivery as such can be replaced (right now there is redundancy with a "static" and "ipfs" copy of each file on the mediator server, which isn't ideal, but this can now be dealt with) Here's a checklist of stuff to do before I would consider it viable to use/test on a publicly-accessible server: https://gitgud.io/threshold862543/orbitchan/-/issues/9
Locutus might be a good platform to build this on. It's basically a continuation of Freenet that supports more dynamic websites. It can be thought of as Ethereum for non-financial uses and without a blockchain, due to it's use of "cryptographic contracts" that govern each site. https://freenet.org/


Forms
Delete
Report
Quick Reply