/hydrus/ - Hydrus Network

Name
Options
Subject
Message	Max message length: 12000
files	Drag files here to upload or click here to select them 0.00 / 50.00 MB Max file size: 32.00 MB Total max file size: 50.00 MB Max files: 5 Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password	(used to delete files and posts)
Misc

What stops you from using hydrus more? Anonymous 06/01/2018 (Fri) 23:17:06 Id: 9f4afb No. 9010 [Reply] [Last]

I think that suggestions make the program better, improvements here and there also go a long way, but really there are hard aspects of the program that cause people to stop everything they are doing till they are improved/reworked. let me give an example. I have an image db that is currently 2 million images large. There was no way in fuck that I was keeping tabs on this shit on its own, so the program is very good in that regard, at least my shit is sorted and can be brought up if I tag/rate something. now, here comes the stop gap. I have 2 million files, and before my db corrupted, I had something around the area of 200k potential matches for duplicates so I need to go through the duplicates first my database broke through so I have only 827k images in the dup finder with 73k images. Now thinking ok, while things get sorted out, why not do them? dup finder only works on 2 images at a time, regardless of if the rest of my db in or not there is work that can be done well here is a random pair

Message too long. Click here to view full text.

50 posts and 2 images omitted.

Anonymous 08/18/2018 (Sat) 09:52:56 Id: 02dfeb No. 9725

>>9493 For #2, dumb-search to smart regex translator? Anyone? Maybe use https://github.com/VerbalExpressions/PythonVerbalExpressions ?

Anonymous 08/22/2018 (Wed) 23:41:19 Id: 620210 No. 9758

>>9481 >>9482 >>9483 Most of this garbage is user error / lack of knowlege. Since others have explained some of your other gripes pretty well, I'll explain custom namespaces. It's SUPER EASY if you took more than a minute to look at the options. Just go to file > options > tag summaries. From there, just select what part of the thumbnail you want to change and you can add whatever namespaces your little heart desires. You can also change their colors so the dark theme won't look as bad (pic related). You can also go to file > options > sort/collect to make custom namespace sorting options for your shitty porn collection, and file > options > colours will let you make your namespaces look pretty.

Not getting one of these Anonymous 10/13/2018 (Sat) 13:47:22 Id: ab9f14 No. 10250

https://www.ebay.com/itm/HP-DL380p-G8-16x-SFF-Intel-XEON-E5-2600-v2-Serverschmiede-Server-Konfigurator/222417019581?hash=item33c91662bd:g:CdgAAOSw11ZZeL8A https://www.ebay.com/itm/Cisco-UCS-C240-M3-16x-SFF-32GB-4x-8GB-2x-Xeon-E5-2660-2-20-GHz-Raid-Server/222438386802?hash=item33ca5c6c72:g:D0gAAOSwuxFYxqFx https://www.ebay.com/itm/HP-Proliant-DL380p-G8-Gen8-Server-16x-2-5-SFF-XEON-E5-2600-v1-v2-P420i-Raid-CTO/222751766786?hash=item33dd0a3902:g:TPAAAOSwZq5bqiIR:rk:5:pf:0 https://www.ebay.com/itm/Cisco-UCS-C240-M3-16x-SFF-8GB-2x-4GB-2x-Xeon-E5-2660-2-20-GHz-Raid-Server/222438386145?hash=item33ca5c69e1:g:OYkAAOSwB-1YxqG6:rk:28:pf:0 https://www.ebay.com/itm/Dell-PowerEdge-R720-16x-SFF-XEON-E5-2600-Serverschmiede-Server-Konfigurator/322381837178?hash=item4b0f74737a:g:1zIAAOSwH9lZeL2y:rk:35:pf:0 https://www.ebay.com/itm/Cisco-UCS-C240-M3-16x-SFF-32GB-8x-4GB-2x-Xeon-E5-2660-2-20-GHz-Raid-Server/222438386453?hash=item33ca5c6b15:g:ftwAAOSwsW9YxqGX:rk:46:pf:0

Version 324 hydrus_dev 09/26/2018 (Wed) 19:46:58 Id: fd51ee No. 10089 [Reply] [Last]

https://www.youtube.com/watch?v=iLnEHzpURXk windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v324/Hydrus.Network.324.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v324/Hydrus.Network.324.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v324/Hydrus.Network.324.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v324/Hydrus.Network.324.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v324/Hydrus.Network.324.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v324.tar.gz I had a great week. The downloader overhaul is almost done. pixiv Just as Pixiv recently moved their art pages to a new phone-friendly, dynamically drawn format, they are now moving their regular artist gallery results to the same system. If your username isn't switched over yet, it likely will be in the coming week.

Message too long. Click here to view full text.

42 posts and 1 image omitted.

Anonymous 10/04/2018 (Thu) 00:49:39 Id: 72d0b8 No. 10190

>>10189 I get that error too. But lz4 is just some obscure type of compression that's not used in any file formats I know, and it doesn't seem to cause problems, so I didn't bother reporting it.

hydrus_dev Board Owner 10/06/2018 (Sat) 19:16:17 Id: fd51ee No. 10205

>>10188 Should be about the same, as the frozen exe is just spinning up a python instance and running the same code, afaik. Boot should be a bit faster since it doesn't have to mess around with that bootstrap environment shit. That's interesting that that page kept the files through the trashed->full delete transition. They should be removed automatically.. That suggests the underlying file context of those pages is 'everything' rather than 'local'. Or maybe they are loading from db subsequently in a non-filtered way. I'll check this, thank you for reporting it! >>10189 >>10190 Thanks. If you figure out any new/unusual AssertionErrors getting put to your log (not the GTK-critical stuff, which doesn't give me enough info, but a proper python/wx error trace), please let me know. There are still some instabilities in the linux code, but I don't always get a good traceback in my environment. lz4 isn't a big deal. I've been unable to package it properly for a few months now due to a PyInstaller update. I burned some hours on it and couldn't figure it out, so I will revisit after the python3 update.

Anonymous 10/07/2018 (Sun) 19:44:36 Id: 72d0b8 No. 10213

>>10205 Okay, I will have a look at running the source then.

What kind of dedication and work does it take to create something like this? Anonymous 09/30/2018 (Sun) 03:53:35 Id: 25b943 No. 10128 [Reply]

I bet you thought this was a thread made to suck on Hydrus Dev's cock. HAHA, WRONG! There's so many things about the Hydrus network that irritate the crap out of me, and it's clear that, as they're deliberate design considerations, they're not going to ever change. Therefore, I want to make my own. Problem is, I haven't really developed shit before, so I just want to know what I'm in for. Maybe I'm mistaken, but I'm under the impression that Hydrus Dev was only starting out when he began creating his as well, so maybe he could offer a few pointers, being already so far into his own project. Anyway, here's some notes I've been making on how I might do things differently. I imagine Hydrus Dev has already considered a lot of these ideas before, so being more world-weary, and having lost that old shine in his eyes, maybe he could tell me why they're fucking stupid and wouldn't work at all. Pyrus Database - Store tags and other filesystem-related metadata in two separate databases for full reversibility. Since I consider reversibility so important, I think it deserves this level of security, to be stored in a more stable database of its own which gets written to less frequently and is therefore less prone to breakage. This secondary database will contain all forms of metadata the source filesystems could possibly support; not just the basics like filenames or creation dates, but also comments music tags, or even multi-purpose "description" fields. Users could also create custom keys such as "Likes" for archived YouTube or Twitter posts. Naturally, this level of data retention may seem redundant to many users, so it wouldn't be mandatory. - Also store the full source paths of programs, or, optionally, just the final directory name. When certain files are "marked", they may all be stored (or at least symlinked) in a single internal directory with all of their original filenames. This would allow certain flash files, and even games or programs to function normally within the database. Naturally, these directories should be excluded when you start the server up. I'm thinking of a hierarchy like this.


  db/.
  db/everything else
  db/symlinks
  db/symlinks/(Games/(Visual Novels/))euphoria
  db/symlinks/(Programs/)Hydrus Network
  * "symlinks" subdirectories up to user discretion, even if they don't seem necessary to me when programs must work with relative paths for this level of abstraction to work anyway.

- No silly client-side case conversions. Come on, I mean we have the full fucking range of the UTF-8 character set here, not 7-bit ASCII. If users really feel like it, then maybe include an OPTION to use this kind of filter. With this and the previous point, some measure of reverse compatability with the Hydrus PTR might be possible, but that sounds like too much of a hassle. - Since community spirit is important for a federated user participation-based booru project like this, maybe even throw in a third database for user ratings and comments. Pyrus Client - Develop a GUI and TUI in tandem, or if it has to be a GUI, at least don't use a terrible library that forces you to look at the hideous default GTK3 theme. - Don't do anything absolutely fucking stupid like develop a garbage "Media Viewer" implement and integrate it so deeply into the operation of the program that other features must get held back for it. - Allow users to create custom execs for each filetype, or even for entirely different database variables. If a user doesn't want to define another set of execs for their client, then they can simply write "rifle"/"xdg-open"/"whatever" into each field. Since we've abandoned the Media Viewer, which is used to untangle the messy jungle of a filesystem behind the curtains, we can go even more abstract and create variables for several client-specific factors to be passed as arguments for the exec. %s might mean all selected files

Message too long. Click here to view full text.

Anonymous 10/03/2018 (Wed) 12:54:42 Id: bf0bb1 No. 10175

>>10128 This is the guy from 8/tech/ telling you to show code or GTFO. > UTF-8 boooo If you do download images from Pixiv you will see why you are flawed. > Moar tags Already on its way to the public > YouTube and Twitter videos On the to-do list already > Symlinks and secondary DB There are winfags here so it is not for everyone… On the to-do list though. > No client-server difference You will regret this once someone implements a 3rd-party client/browser/viewer to work with the upcoming API > the hideous GTK3 theme WxPython for Winfag compatibility. Move it to Qt if you have balls. > No "media viewer", rely on VLC/MPV On the to-do list

Message too long. Click here to view full text.

Anonymous 10/04/2018 (Thu) 23:32:45 Id: 25b943 No. 10194

>>10175 > UTF-8 boooo I'll forgive you for misinterpreting my rant, but that was about characters being converted to their lower-case forms, not something needless like converting unicode to hex. Considered the rest though, maybe I'll wait and see how good Hydrus gets.

Anonymous 10/05/2018 (Fri) 09:36:04 Id: a6d673 No. 10196

>>10194 "Talk is cheap, Show me some code" Linus Torvalds "Release Early, Release Often, Listen to your customers" Eric S Raymond "No problem is too big it can't be run away from" Linus Torvalds "Prototype then polish, Get it working before you optimize it" Eric S Raymond "Any program is only as good as it is useful" Linus Torvalds "With enough eyes, All bugs are shallow" Eric S Raymond "Those that can, do, Those that can't, complain" Linus Torvalds

Release Tomorrow! hydrus_dev 10/02/2018 (Tue) 23:42:55 Id: 641b54 No. 10169 [Reply]

I had a difficult week, but I got some great work done. The downloader overhaul is entirely complete but for some help. Tomorrow's release has a variety of unusual new features, such as subscriptions without popups, subscription 'quality' review, improved easy-import download pngs, more download logic controls, improved file system healing, and more. The release should be as normal tomorrow.

Anonymous 10/03/2018 (Wed) 00:36:58 Id: 117966 No. 10171

Any chance you'll address my complaints?: >>10160 >>10161 I can keep patching it myself, but I feel like a relative update interval for high-velocity threads ranging from 5 minutes to 10 seconds is something reasonable (especially because it's not a problem for some imageboards github.com/4chan/4chan-API).

Anonymous 10/03/2018 (Wed) 08:51:19 Id: cca655 No. 10174

Nice. That certainly sounds interesting. BTW, not sure if you saw it already but Pixiv did recently change the profile pages. Doesn't seem to completely break the downloader, but it seems to result in some failures. Apparently on the first 4 or so downloads it tries.

hydrus_dev Board Owner 10/03/2018 (Wed) 18:51:01 Id: 641b54 No. 10183

>>10171 Thanks, answered in main release thread. >>10174 Thank you for reporting this–clearly the Nips are at it again. I don't have time to look into it for today's release, but I will check back on Saturday.

Version 316 hydrus_dev 08/01/2018 (Wed) 21:40:07 Id: 615bc5 No. 9533 [Reply] [Last]

https://www.youtube.com/watch?v=GKpKvnSjOtU windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v316/Hydrus.Network.316.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v316.tar.gz I had an excellent week. I was able to cleanly 'do' the big gallery overhaul, resulting in many changes. This is an important release for anyone who does a lot of downloading. gallery update So, the meat of the gallery overhaul is complete! Where available, subscriptions, the gallery downloader, and the url downloader will now use the new parsing system to parse 'thumbnail pages'. The new pipeline has numerous improvements behind-the-scenes and is another important step in allowing users to create and share their own downloaders for new sites.

Message too long. Click here to view full text.

39 posts and 1 image omitted.

Anonymous 10/01/2018 (Mon) 04:27:39 Id: 92090e No. 10150

>>9579 >>9580 >>9581 >>9586 >>9589 Let me get this straight. You people LIKE vim? Hell, I use it every so often, but only if I need to make a super-minor change and don't feel like waiting for a graphical text editor to load. Or, more likely, it can't load because I FUBARed my system and need to fix it. Hell, I've always thought that all of those people talking about and comparing text-only editors were either old farts who worked on mainframes in the 1980s or people just joking around. I'm not arguing against improvements, mind you. I'm not even arguing about priorities. I'm just confused as all hell. >>9587 Low-resource mode is a great idea. With the old download system, I've long wished to be able to somehow page out only part of the memory Hydrus was using (yes I know paging doesn't work like that, but let me dream goddammit). At one point I actually bought more RAM so Hydrus worked better, especially when also doing something with my computer that's not Hydrus.

Anonymous 10/01/2018 (Mon) 06:23:40 Id: 92090e No. 10152

>>10150 Shit, I thought this was the new thread.

Anonymous 10/02/2018 (Tue) 05:17:01 Id: 3d1991 No. 10159

>>10150 >I'm just confused as all hell. Why? There is a lot of people who uses vim as their primary text editor, people really really like it, there is even a game to learn Vim, the community is kinda like Emacs (but I think Emacs users are more dedicated to their text editor), there is a lot of programs that use vim or emacs shortcuts, one example is the Terminal, by the default they use the Emacs shortcuts but you can set it to use Vi shortcuts, I prefer to only use the keyboard so most of my programs use the vi keys (I use them even in my web browser) , I try to do the same with hydrus but it ins't completely possible at least yet.

Version 323 hydrus_dev 09/19/2018 (Wed) 22:38:26 Id: e3ce6d No. 10032 [Reply]

https://www.youtube.com/watch?v=3k5KWFnnXK4 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v323/Hydrus.Network.323.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v323/Hydrus.Network.323.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v323/Hydrus.Network.323.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v323/Hydrus.Network.323.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v323/Hydrus.Network.323.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v323.tar.gz I had a good week. I finished a decent first version of the downloader easy-import and did some other stuff. easy import There is now a simple new dialog under network->downloaders->import downloaders that lets you import new downloaders just by dragging and dropping an encoded png onto it. It tries to do all the semi-complicated comparison and association work to actually build a new downloader clientside automatically.

Message too long. Click here to view full text.

29 posts and 1 image omitted.

hydrus_dev Board Owner 09/26/2018 (Wed) 14:54:16 Id: e3ce6d No. 10087

>>10071 >>10069 >>10072 >>10076 Thanks. Although your threads are spiking there, it doesn't look horrific or out of place. My guess is this is ui-bound, particularly since looking at the busy watcher page is making things noticeably more laggy. I will review this code and see if I can optimise and slow the refresh rate based on the number of watchers being tracked. Although the client uses different threads to do its stuff, python unfortunately cannot use multiple processor cores simultaneously, so it never really is 'truly' multi-threaded. It can use multiple when it dips down to the C++ level (typically when it is doing low-level stuff like heavy math/file processing), but otherwise, it'll typically max out at 12.5% or 25% CPU. My threading code sucks as well (it does a lot of async shit synchronously, just waiting on results), which is a big cause of a lot of ui hang.

Anonymous 09/26/2018 (Wed) 15:33:43 Id: 6e8a57 No. 10088

>>10087 One of the things you responded to was a different issue with a subscription. On the side of reducing load. If possible can you keep the top level things, as I like seeing XXX/XXXX to see how much work is left. Sometime this week im going to dump the entirety of thread backlogs into a saved session, I have a feeling the watchers are causing quite a bit of ram use even if they aren't visible, it will at least give me a point of comparison once I dump the somewhere around 8000-10000 threads watchers down to around 800 active.

Anonymous 09/27/2018 (Thu) 14:50:11 Id: 12ad01 No. 10093

>>10085 I'm running from source, specifically the AUR package from Arch Linux. Thanks for looking into it, and I'll run in it report mode when the time comes.

Version 322 hydrus_dev 09/12/2018 (Wed) 22:42:38 Id: a3742c No. 9943 [Reply] [Last]

https://www.youtube.com/watch?v=80uElI8gtD8 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v322/Hydrus.Network.322.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v322/Hydrus.Network.322.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v322/Hydrus.Network.322.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v322/Hydrus.Network.322.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v322/Hydrus.Network.322.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v322.tar.gz I had a good week. I took it a little easy after pushing to get over the last big overhaul hump last week. There's some fixes, new help and interesting misc work. downloader help and fixes I am thankful and relieved that last week's big shift seems to have basically gone well. There were a couple of bugs–artstation and tumblr were only searching the first page of their galleries, and the new GUG object had some unclean ui in places–but these are fixed today. Working with that code without having to deal with the old legacy system has been great.

Message too long. Click here to view full text.

56 posts and 1 image omitted.

hydrus_dev Board Owner 09/19/2018 (Wed) 20:09:44 Id: a3742c No. 10028

>>10019 1 - No. Other than bmps (which are automatically converted to png on import) hydrus should keep all files exactly the same, byte for byte. 2 - Yeah, should do these days. At some point, this info should be cached and searchable. Accessed will change any time you look at it, obviously. 3 - When I add that cache, I'll probably add it as a new kind of metadata, rather than explicit tags, which don't store specific timestamps efficiently, and search for it with a new system predicate, like 'system:time imported', but 'system:file creation/modified time'. If you want to search this info, just keep importing as normal–since it is preserved through import, no information is being lost, and when I make the cache, I'll retroactively fill it with existing file data.

hydrus_dev Board Owner 09/19/2018 (Wed) 20:11:33 Id: a3742c No. 10029

>>10023 Thank you for this report. I am still moving all my old shortcut code to the new system. Mouse shortcuts are not well supported at all yet–the code is just not yet in to handle it. I hope to have more time to put into this as the downloader overhaul comes to a close.

Anonymous 09/19/2018 (Wed) 22:02:51 Id: 4987fd No. 10031

>>10029 Thanks. That will be awesome.

Release Tomorrow! hydrus_dev 09/11/2018 (Tue) 22:35:58 Id: 35524f No. 9937 [Reply]

I had a good week. I fixed some bugs in the new download pipeline (including the artstation/tumblr short-search issue), caught up on a bunch of unusual small jobs, and finished the bulk of the 'how to write a downloader' help docs for those users interested in trying it out. The release should be as normal tomorrow.

Anonymous 09/16/2018 (Sun) 23:28:38 Id: 3caf5d No. 10010

I just want you to know I appreciate you.

hydrus_dev Board Owner 09/19/2018 (Wed) 22:00:30 Id: 35524f No. 10030

>>10010 👌

Version 321 hydrus_dev 09/05/2018 (Wed) 21:31:52 Id: e19786 No. 9891 [Reply]

https://www.youtube.com/watch?v=rS9Tgg_R1M8 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v321/Hydrus.Network.321.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v321/Hydrus.Network.321.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v321/Hydrus.Network.321.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v321/Hydrus.Network.321.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v321/Hydrus.Network.321.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v321.tar.gz I had a great week. An important milestone in the downloader overhaul is done. Users who use 'custom' boorus will want to read this post and may like to put off updating a week or two. downloader overhaul So, this week I have switched over the old hardcoded method of converting search text into an initialising gallery url to the new user-customisable and -shareable GUG (Gallery URL Generator) objects. This was the last hardcoded component I needed to replace, and so I was finally able to disconnect and delete all the old downloader code. We are now entirely running on the new customisable and shareable system!

Message too long. Click here to view full text.

22 posts and 1 image omitted.

Anonymous 09/12/2018 (Wed) 02:54:31 Id: 1925e5 No. 9938

>>9934 You could make this public so people can write better reports on new features, you know?

hydrus_dev Board Owner 09/12/2018 (Wed) 17:40:21 Id: e19786 No. 9942

>>9938 I'm afraid that's my point–I know I am bad at dealing with collaborative systems. I find it just becomes talking talking talking and stresses me out to the point of abandoning the whole platform and the users along with it. This was most recently true of Github Issues, which I ended up turning off because I couldn't keep up with it enthusiastically. Like many other Anons, I'm a complete fucking sperg, so figuring out what workflow lets me put the keyboard work in week to week without imploding in drama has been an important part of the whole deal for me. Although the current to-do is a mess, I can't see a more productive system for me atm. I appreciate your point, but for me the benefits are outweighed by the inevitable personal drawbacks. I'm open to other solutions, if you can think of them. Some users on the discord put together platform where they could discuss new features and so on–which I had no problem with, although I did have this same convo with them–but I don't know whether it actually succeeded beyond the first two fun weeks. If you are interested in this stuff, you might like to chase down whatever happened to it.

Anonymous 09/14/2018 (Fri) 14:55:49 Id: c24889 No. 9969

>>9891 Hey dev, The Derpibooru integration works pretty well, but seems to be subject to the site's default filter which hides anything not kid-friendly. You should request no filtering by passing `&filter_id=56027` in with each call to the search endpoints (assuming showing everything is a reasonable default). The filter_id parameter is mentioned briefly on the API docs page [0], and the ID is from the Everything filter on the filters page [1]. [0]: https://derpibooru.org/pages/api [1]: https://derpibooru.org/filters

Tag parser script Anonymous 01/29/2016 (Fri) 04:09:28 Id: d175d6 No. 1914 [Reply] [Last]

Here's something I did to make copying individual booru tags easier. hydrus is good with booru browser but its page downloader doesn't read tags. And sometimes you just wanna quickly add tags to an individual image. So this tamper/greasemonkey script will add a button and a key bind to booru posts and the Illustration2Vec demo site to copy all tags on the page including their schemas and rating. Then you can just right click your image in hydrus > manage file tags > paste tags > apply. The whole process takes about 5 seconds. github.com/JetBoom/boorutagparser

51 posts and 1 image omitted.

Anonymous 08/25/2018 (Sat) 09:27:22 Id: 1bb151 No. 9780

>>9703 Sure it's possible, but you'd have to edit the script. I did some minor changes myself but I'm not too familiar with the code/language so I couldn't do it for you without some effort.

Anonymous 08/27/2018 (Mon) 17:25:39 Id: 1f5ecd No. 9810

>>9780 Do you perhaps know how to at least make it download the original filename? My idea of grabbing the post number was probably overkill, with danbooru and yande.re it'd just be fine to have the original name, but i have no idea how do edit the script to tell it to stop the renaming function I mean, i think i've even found the line in the script with "var filename", but i don't know what to do with it

Anonymous 09/05/2018 (Wed) 17:48:43 Id: a2cc1d No. 9890

Tag parser doesn't get meta tags currently, any plans for this?

Important Release Tomorrow! hydrus_dev 09/04/2018 (Tue) 23:04:17 Id: 6731c2 No. 9885 [Reply]

I had a great week. A big milestone in the downloader overhaul is done: all downloads, from start to finish, now happen on the new system's code. Any advanced user can create and share a completely custom downloader for any site the new parser can figure out. I am really pleased with it. There are some final important objects to update. Most users will hopefully experience no big changes, but anyone who uses a custom booru (that they created or imported through the manage boorus dialog) will want to read the release post before they update. There will be nice new downloaders for Artstation and Derpibooru, and the Newgrounds downloader makes a return. I've got a bit more to test, but the release should be as normal tomorrow.

Anonymous 09/05/2018 (Wed) 15:09:40 Id: 8edc36 No. 9887

>>9885 Praise The Lord!

Anonymous 09/05/2018 (Wed) 15:29:52 Id: 8edc36 No. 9889

https://invidio.us/watch?v=Z4DqCevajw4 Current Mood. The time has come. Ascension into godhood

Version 320 hydrus_dev 08/29/2018 (Wed) 21:41:18 Id: ce499d No. 9822 [Reply]

https://www.youtube.com/watch?v=oWl-xtan44o windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v320/Hydrus.Network.320.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v320/Hydrus.Network.320.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v320/Hydrus.Network.320.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v320/Hydrus.Network.320.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v320/Hydrus.Network.320.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v320.tar.gz I had a great week. The downloader overhaul is in its last act, and I've fixed and added some other neat stuff. There's also a neat hydrus-related project for advanced users to try out. Late breaking edit: Looks like I have broken e621 queries that include the '/' character this week, like 'male/female'! Hold off on updating if you have these, or pause them and wait a week for me to fix it! misc

Message too long. Click here to view full text.

28 posts and 1 image omitted.

Anonymous 09/02/2018 (Sun) 03:49:23 Id: fa52c8 No. 9867

>>9851 Based, it worked. ty

hydrus_dev Board Owner 09/02/2018 (Sun) 15:28:22 Id: ce499d No. 9869

>>9866 Thanks. I'll check out the spammy popup on missing location. I'll be revisiting file import options in the nearish future to do some url- and hash-checking logic options. While I am in there, I'll see if I can add an option for an artificial 'wait x seconds after every import' to throttle things.

Anonymous 09/03/2018 (Mon) 03:29:32 Id: 1abef3 No. 9872

A web interface for the server? Hell yes! Hopefully, I will be able to figure out how to set up all that crap. Being able to host my own little Booru from my Hydrus db is something I wanted to do for a while now.

Version 319 hydrus_dev 08/22/2018 (Wed) 22:01:01 Id: ef894a No. 9757 [Reply]

https://www.youtube.com/watch?v=8bw5h-WPYBQ windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v319/Hydrus.Network.319.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v319/Hydrus.Network.319.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v319/Hydrus.Network.319.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v319/Hydrus.Network.319.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v319/Hydrus.Network.319.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v319.tar.gz I had a mixed week, but I got some good work done. It is mostly improvements to the downloaders, and then some stuff for advanced users to play with. downloaders and urls Following the success of the new gallery page 'delay' system, where gallery page hits across the program are spread out over time, I have made several improvements: the system's logic is more reliable, it occurs on a per-domain basis, the waiting period and status updates now occur in the network engine itself (so will appear in the network job control ui–I expect to add some kind of 'override wait' action to the cog menu soon), and the thread watcher now also has a delay similar system, also under options->downloading. Also, the network engine will now not permit more than three simultaneous jobs per domain as a global bottleneck, and the overall total num connections is boosted as a result to 15 (from 10). In general, the network code should be just a bit more pleasant and less CPU-spiky while still serving a decent throughput when there is work to be done.

Message too long. Click here to view full text.

20 posts and 1 image omitted.

hydrus_dev Board Owner 08/28/2018 (Tue) 21:14:08 Id: ef894a No. 9812

>>9809 I happened to get this error myself this week and think I figured it out. Please let me know if you have any more trouble in v320 on!

Anonymous 08/29/2018 (Wed) 04:05:02 Id: dd22c0 No. 9819

I found a cool bug in v318, but not sure what it is or if you already know about it. Here goes: >I had several multiwatcher tabs open, most idle but one downloading 12 or so threads at once >I had one simple downloader tab open, downloading from around fifteen threads from an off-site archive >also syncing tumblr subscriptions at same time >switch to the dupes tab and run the rebalancing, hadn't done it since before the new tumblr fuckery so I had I think 84 branches to rebalance >goes for a while, I'm tabbed out reading a thread and messing with jdown and stuff >tab back in, it's just finished, hit OK, instead of letting it sit a minute and then running dupe discovery, I immediately switch back to the working multiwatcher, but hydrus hangs as it tends to because my laptop sucks, especially after finishing big operations, a message box has popped up under the subscriptions box, but it's blank because the client froze there >after ten or twenty seconds it resolves itself and refreshes the screen, seems normal, except the subscriptions downloader box and the message box below it are just gone >figure it was just a message saying it thought it was done syncing for the day, not sure why they closed themselves though >elapse ten or twenty seconds of normal behavior >open up tags to enter in new tag for thread I'm about to paste into multiwatcher, enter them and remove last thread's unique tags, hit OK or Apply or whatever, box closes >client hangs again, this time doesn't recover, but rather than going (Not Responding) it just makes that annoying system bell whenever I try to click on any tab or anything at all >downloader pops back up and starts, figure it's fine and I can just wait a bit since it's actually downloading >stays frozen, can't select anything, but downloader still going and there's a rare C++ level error spat out at me instead of Python >running it in debug, so go to console window and look at it, shows basically same thing (forgot to screenshot the full error, sorry) >was able to select the program itself before from other front windows, but after selecting the debug console now can't select the GUI and the console stays up in front, still dinging >Ctrl+C from the console so it doesn't crash my computer when I try to close the GUI

Message too long. Click here to view full text.

hydrus_dev Board Owner 08/29/2018 (Wed) 16:55:20 Id: ef894a No. 9821

>>9819 Thank you for this report. This looks the same as >>9782 . The 'modal' popup that comes up during idle maintenance and a couple of other places was being rude to any pre-existing dialogs. The dialog gets hidden but then can't restore when the maintenance job is over, and since the dialog is still modal, ui functionality can't return to the main ui frame. In your case, it was probably the tags window for your new multi-watcher. Maybe some idle job got delayed while the dupe search was going on, arrived late once your db-commit hang finished, and then blatted itself into the ui event loop right when you had that dialog open. I believe I have this fixed for today, but if you experience it again, particularly with the dupe search stuff–which works on slightly different logic–please let me know. I now know what the error is actually about!

Version 318 hydrus_dev 08/15/2018 (Wed) 21:39:12 Id: ea9080 No. 9695 [Reply]

https://www.youtube.com/watch?v=ZQSpPrZSDQ8 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v318/Hydrus.Network.318.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v318/Hydrus.Network.318.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v318/Hydrus.Network.318.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v318/Hydrus.Network.318.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v318/Hydrus.Network.318.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v318.tar.gz I had a great week. I caught up on more small stuff and put some more work into the new gallery downloaders. downloaders Unfortunately, tumblr decided to kill their 'raw' url access this week. There is not a lot of firm word from tumblr on the issue, but there is some scattered conversation around, and it seems it is dead and not to return. Whatever the ultimate reason for this, it broke our tumblr downloader, so v318 has an updated tumblr parser to fetch the 1280px versions of urls. I have also taken the opportunity to switch the tumblr 'gallery' parser over to the new system, so the tumblr downloader now fetches and associates neater 'post urls' for its file import objects rather than raw file urls and adds 'creator' tags. However, because of the URL changes, your tumblr subscriptions will hence hit their 'periodic' file limits and likely redownload some 1280px versions of files you already have in raw–if this affects you heavily, you might want to pause your tumblr subs before you update and carefully experiment and curate what happens after you are working again in v318. Note that some artists (like lmsketch) attach a lot of images to their posts, so if your periodic limit were 100–and that 100 now means 'posts' instead of file urls–you are potentially talking a lot of files that are going to be redownloaded! Again, I recommend heavy tumblr subscribers pause before update and maybe consider recreating their tumblr subs from scratch with an initial file limit of 10 or so.

Message too long. Click here to view full text.

25 posts and 1 image omitted.

Anonymous 08/20/2018 (Mon) 11:23:13 Id: 15e1fa No. 9745

>>9742 would love to know who it was.

hydrus_dev Board Owner 08/25/2018 (Sat) 19:14:13 Id: ea9080 No. 9784

>>9735 I don't have that in my version, but I think I remember maybe turning something like that off when I was using it more. There's some auto-backup options somewhere, maybe there are some indexing options near there? Like you can tell it not to check for updates to file structure or something. That shit was one reason why I started thinking about hydrus, actually. The difficulty of keeping up with a disparate file structure. >>9737 Yeah, looks like e621 has several examples of these unreliable source urls. I will add options to file import options to change the url check logic here (e.g. '[] don't trust source urls for 'already in db/deleted' checks). >>9744 Yeah, it is all gone. The amazon links still respond in different ways, suggesting the 'bucket' still exists, but you get variants of 'authorisation missing'. I presume they have basically tightened up their AWS-side back-end CDN to require authentication, so it is back to being a way for their cache servers to grab originals. tumblr are aware of the issue and have strongly suggested it is never coming back, so I imagine if we ever find a new way back in, they will only shut it down again. This is happening all over the place. My feeling is the future of original source is going to be the various boorus 'inheriting' from the actual original source, be that patreon or some tumblr post that links to HD on imgur or mega or whatever (although if the boorus use CloudFlare, they may be compressing jpegs on the fly! fuck!). In five years, I expect hydrus to have P2P file-sharing components. You can trust that any system I make wouldn't do this shit. But I can also imagine more complicated waifu2x-style systems in future that may nullify most of the issue.

Anonymous 08/27/2018 (Mon) 07:46:18 Id: eec8b9 No. 9806

>>9745 A furporn Patreon artist

Version 317 hydrus_dev 08/08/2018 (Wed) 21:19:51 Id: 744617 No. 9608 [Reply] [Last]

https://www.youtube.com/watch?v=pE64C2RX3ns windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.OS.X.-.App.dmg tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.OS.X.-.Extract.only.tar.gz linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v317/Hydrus.Network.317.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v317.tar.gz I had a great week. I polished some ongoing download stuff and caught up on a bunch of smaller jobs. tag filter and tag import options A while ago, I wrote a new 'tag filter' that applies blacklist and whitelist rules to a list of tags. It is more powerful than some of the old systems I was using at the time, but the ui was pretty ugly, so I didn't roll it out everywhere it could be. This week, it gets new responsibilities and completely reworked ui and workflow.

Message too long. Click here to view full text.

52 posts omitted.

Anonymous 08/15/2018 (Wed) 22:15:35 Id: 244d75 No. 9696

>>9693 The way that I see it, and less so from dan then gel or other boorus, it seems like artists can have multiple names and get tagged that way, however the thing is most boorus, if one has multiple names for an artist, you can bet that everyone does. so lets say I had a rule for all the r34 pages and gel, and I want to search an artists name, lets say Hereismyname So I put it in there I find out he has a second name Nothatassholeislyinghereismyname so I do it again, and finally I find out they go under an alias for some art Fuckyouthisismyname its likely each one of those is going to be a hit, so instead of entering those each once for each one on their own, hitting them all at once with each name would be a better option. The only one this wouldnt work for is derpibooru, as derp does an annoying thing where to search an artist you have to type artist: to search artists, rather then just their name, and if you search by name you are never finding them. but on the topic of nesting, would it be possible to make a collapsible nest? something kind of like this, sure its for folders by this could work similar. nvme boot could be artist search brother could be an artists name which could be expaned to show its source nvme boot could open all the images under it in one tab

Message too long. Click here to view full text.

Anonymous 08/15/2018 (Wed) 22:32:35 Id: 244d75 No. 9697

>>9685 ok I have it and its in, though not 100% sure how to use it Ok, got it in the parser, it still returned a gif Ok, thinking there was a conflict, I deleted the original parser after exporting it, Now I get there are no parsers Ok, import the original parser again No parsers Delete the new one No parser I think I did something stupid and i'm not seeing how to fix it/enable a parser do I have to restart the program each time or am I just doing something really wrong and dont realize it?

Anonymous 08/15/2018 (Wed) 22:58:03 Id: 31e267 No. 9699

>>9686 The file sizes look fine and I have all 4. I did the cloning/renaming correctly but only did client.db, that's the only one it said in the help txt. Should I do the other 3?

Release Tomorrow! hydrus_dev 08/07/2018 (Tue) 22:21:34 Id: 8a5d48 No. 9593 [Reply]

I had a great week. I have completely revamped some important tag filtering ui, polished the new multi-gallery and -watcher, caught up on a bunch of older issues, and added parsers for imgur and derpibooru. This release should be as normal tomorrow.

Anonymous 08/08/2018 (Wed) 03:54:16 Id: 4936fd No. 9596

fantastic, Just get hydrus down to a reasonable image size and then you add derpi… hello 600k+ images again. cant wait

Anonymous 08/08/2018 (Wed) 05:03:18 Id: fdf70d No. 9597

>>9593 Getting Fiber in next few weeks… looks like I should get a server/NAS for Hydrus!