/hydrus/ - Hydrus Network

Name
Options
Subject
Message	Max message length: 12000
files	Drag files here to upload or click here to select them 0.00 / 50.00 MB Max file size: 32.00 MB Total max file size: 50.00 MB Max files: 5 Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password	(used to delete files and posts)
Misc

Anonymous 12/13/2019 (Fri) 14:28:16 Id: 120bcd No. 13456 [Reply]

thank you mr. hydrus for the software now i can download my furry porn from 8kun without having to fix my old scraper

Anonymous 12/13/2019 (Fri) 14:31:04 Id: 120bcd No. 13457

ok nevermind it turns out that it does not work with 8kun

Anonymous 12/14/2019 (Sat) 23:21:41 Id: 30b621 No. 13462

>>13457 Yeah, unfortunately the DDoS protection is a block for now. You may have luck using Hydrus Companion to copy your browser's cookies across to hydrus, but I don't know if this will work for 8kun specifically. https://gitgud.io/prkc/hydrus-companion Once the whole situation calms down, I will roll out what I can to get hydrus working again with 8kun and the popular bunkers.

Version 375 hydrus_dev 11/28/2019 (Thu) 01:59:00 Id: 94a0a3 No. 13437 [Reply]

https://www.youtube.com/watch?v=M5DqEHxY4FM windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v375/Hydrus.Network.375.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v375.tar.gz I had a great week. There are a bunch more Qt fixes, and a few other things as well. Qt I have fixed a bunch more bugs in the Qt code. We are getting to the end now–this is mostly smaller stuff like an unusual dialog button not working, but I have fixed another important memory leak that was causing some backend not to be deleted correctly when a media viewer closed on a video. This should radically reduce memory use for some heavily used clients.

Message too long. Click here to view full text.

3 posts omitted.

Anonymous 11/28/2019 (Thu) 22:06:25 Id: 502ec9 No. 13441

Curiously this thread only shows up in the catalog. At least for me.

Anonymous Board Owner 12/01/2019 (Sun) 02:31:47 Id: d48852 No. 13442

>>13441 Yeah, I have had to force-refresh a couple of threads on the site to get new posts, and a couple of the BO things didn't work immediately after migration. And things are obviously sluggish. Seems to be getting better though! My only real peeve is that file downloads now don't have the content-length header sent or something, so my web browser doesn't have a nice progress bar as they download. And the anti-DDoS blocks hydrus html for now unless, I assume, you send the vanwatech cookies with Hydrus Companion or similar. JSON API seems to work, so maybe that is better. I keep meaning to do this job but keep not getting around to it, but I'll roll out new thread watcher support for the big bunkers, and 8kun, in a release soon.

Release Tomorrow! Anonymous Board Owner 12/04/2019 (Wed) 05:10:32 Id: 7a4da2 No. 13443

I had an ok week. Subscriptions now operate more efficiently, using less resources and choosing their sync times more precisely. Some simple Qt theming support is added for experimentation by advanced users, and a number of bugs are fixed, including page tab drag and drop for macOS users and anyone else with a center-aligned tab bar. I have a busy day, so the release will be quite late tomorrow.

Version 362 hydrus_dev 07/31/2019 (Wed) 22:53:48 Id: bf06a9 No. 13387 [Reply]

https://www.youtube.com/watch?v=PXAlznKcJvA windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v362/Hydrus.Network.362.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v362/Hydrus.Network.362.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v362/Hydrus.Network.362.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v362/Hydrus.Network.362.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v362.tar.gz I had a mixed week. The duplicates overhaul work is finished. duplicates work finished The duplicates storage overhaul is done! Everything is now on the new storage system, the duplicate filter has had some quality of life attention, and now there is some updated help:

Message too long. Click here to view full text.

14 posts and 1 image omitted.

hydrus_dev Board Owner 08/04/2019 (Sun) 19:10:50 Id: 14b112 No. 13429

>>13417 Not yet, but as it happens I am on the verge of doing some heavy work on local tags and I hope to add it then. I'll do a couple of weeks on some audio metadata stuff and then start on all that.

Anonymous 08/04/2019 (Sun) 20:21:30 Id: 5e6830 No. 13433

>>13425 I had added the lookup script for Deep Danbooru but couldn't get it to work; it would get stuck in the uploading stage and quietly fail out. But the errors didn't pop up till maybe 20 minutes after my various attempts so I wasn't sure if they were related or not.

Anonymous 08/05/2019 (Mon) 05:14:41 Id: b5f746 No. 13434

>>13426 Like I said I think, the reason this one works for the jpeg being larger is due to the text, at least that what I think. to get clean text on a png, it's more or less par for the coarse and doesn't eat much space, but for a jpeg, to keep it without any kind of flaw, at least visually, it would need a stupid amount of resources. edge cases like this, or potentially others like this where a jpeg is made, and my best guess is one of the joi threads on 4chan had someone blanket convert with the highest quality settings on them, hit this image and it tossed out a larger file then put in. i'm guessing that 2/3 or above of the image being something jpeg does not handle well had a part in why this exists the way it does. as for an auto decision, I don't see how the difference between 'the pixels are the same choose jpeg' or 'the pixels are the same choose lowest file size' either way, 99%+ of the time it will go jpeg anyway, this would just introduce a safety net for the sub 1% of the time jpeg isn't the smallest choice.

Release Tomorrow! hydrus_dev 07/24/2019 (Wed) 00:41:06 Id: 2e1689 No. 13278 [Reply]

I had an ok week. I did some of the last duplicates work, including matching zooms for files with different resolution ratios and adding a neat pixel-for-pixel duplicate detector, polished some of the logic for the new tag autocomplete search, and cleaned and fixed a bunch of code. The release should be as normal tomorrow.

13 posts omitted.

hydrus_dev Board Owner 07/29/2019 (Mon) 19:42:36 Id: b2bbc6 No. 13367

>>13329 Great. Although the new downloader system is amazing compared to the old rubbish I had, it is not always pleasant to work with with the duplicate objects and so on. The next iteration here will have versioning or something and auto-updates. >>13331 >>13332 I hope to add namespace siblings in some tag repository work that will start in the next 3-5 weeks! This has long been requested. >>13348 Yeah, hydrus performance bombs out at about 30-50k thumbs in a single page. And manage tags doesn't work well for selections above about 8k thumbs. I am afraid the code just can't handle this stuff yet. Next step here is to improve manage tags so it does big operations asynchronously with a nice cancel button, and then yeah, I'd like some sort of faster tech for various large operations overall. The main problem for manage tags is when you try to add a tag, it needs to check all the files in the selection if they already have the tag in order to present you with 'pend to 3 files/petition from 2 files' kind of decisions. It does not scale well (and I think the final 'approve' stop is O(n²)), so for 10k files it just takes an unreasonable amount of time. As well as code optimisations, some CPU-faster entry actions (like 'woah lad, there's a lot of files here, let's make some add-only decisions beforehand') will come at some point. >>13354 Great!

Anonymous 07/30/2019 (Tue) 05:39:56 Id: 8eec11 No. 13372

>>13367 Thanks for taking the time to read and respond to my random feedback on trying to add "pixiv work" tags to those that missed them the first time around. I wanted to say too though that actually I was mistaken- the "url downloader" page only added "pixiv work" tags to the images tagged "r-18". I was able to tell as much because I found a western artist who used to tag his images "r-18g", which is the gore tag, rather than mere "r-18" to mean adult. So every one of his images before he started using the "r-18" tag still didn't have a "pixiv work" tag. I think I may not have a meaningful way to add "pixiv work" tags to every image that failed to get it the first time, in the end. Because even if I manage to do the "url downloader" for as many popular tags as I can, there will always be images that weren't tagged correctly. But the only reason anything even has an "r-18" tag is because I used a "gallery" tab and put in a pixiv member id. I thought I could use the fact I've done that to fill in anything missing a "pixiv work" tag somehow.

hydrus_dev Board Owner 07/31/2019 (Wed) 17:33:05 Id: a4df90 No. 13386

>>13372 Hmm, that's odd. I just did a test for this: https://www.pixiv.net/member_illust.php?mode=medium&illust_id=75961157 As in pic related, and it looked ok. The actual json parsing rules for pixiv work tag look sensible and don't seem to rely on some clever lookup path that goes through r-18 gubbins anywhere. I am not sure. I am not a big pixiv guy and didn't write the new parser, so I can't talk too cleverly. This is a small thought, but is there any chance some of those files that couldn't take a 'pixiv work' tag actually had it deleted in the PTR (if you are adding to PTR)? If you hit F3 on one of them that you know didn't work and then hit the cog icon on manage tags and tell it to show deleted, is there 'pixiv work:123456 (X1)'? I think I have deleted some before after some large petitions came up on the PTR regarding some of this 'meta'-style content. The 'what to do when a tag is deleted' workflow has always been a bit shaky. I'd like to clear some of it up when I move onto this PTR work in 2-4 weeks with better overall local control of what tags you see from difference sources, moving solutions to 'I like this/don't like this' preferences away from the existing 'this is correct/incorrect' workflows.

Version 361 hydrus_dev 07/24/2019 (Wed) 22:13:33 Id: a692a4 No. 13293 [Reply]

https://www.youtube.com/watch?v=v6qckMHp7wU windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v361/Hydrus.Network.361.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v361/Hydrus.Network.361.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v361/Hydrus.Network.361.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v361/Hydrus.Network.361.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v361.tar.gz I had an ok week. Some final duplicates work is done, and there is some polishing to the new tag autocomplete. duplicates The duplicates filter can now detect if static images with the same resolution are pixel-for-pixel duplicates! If they are, it gives one of the standard 'comparison' statements on the right hover window. Furthermore, if one file is a png and the other not, this statement will colour green/red and bias heavily to the non-png, since the png is likely a bloated 'clipboard' duplicate that you don't want. Pixel summary data is not cached long-term, so this routine takes a bit of extra CPU. It only kicks in if both files are images with the same res, but nonetheless please let me know if this makes the duplicate filter too laggy for you. I expect the new 'pixel hash' data will be cached at the db level in future to auto-resolve png/not-png dupes like this.

Message too long. Click here to view full text.

19 posts and 1 image omitted.

Anonymous 07/30/2019 (Tue) 13:39:42 Id: 780cd7 No. 13373

is there a way to filter 'pixel for pixel' duplicates? would love to be able to get those hammered out and could do that very fast too.

hydrus_dev Board Owner 07/30/2019 (Tue) 23:36:07 Id: 89b8d8 No. 13378

>>13373 Not yet–this data is currently generated on the fly in the duplicate filter. But I think it is very worth caching it in the db at some point. I can search it incredibly quickly to find pixel dupes and get started on some auto-dupe-resolve rules like 'if two files have the same pixels and one is a png and the other is not, delete the png' so we can finally and automatically clear out Clipboard.png trash dupes. It would be fast enough that I could even add a hook for it on import! This would be optional and probably default on. More complicated stuff like 'if two jpegs are the same, keep the one with smaller/larger filesize' would be default off and up to the user.

Anonymous 07/31/2019 (Wed) 15:36:48 Id: 780cd7 No. 13384

>>13378 I would set both rules to delete the larger file size, it's very rare, but I have come across some jpegs were saving as png did lower file size.

Release Tomorrow! hydrus_dev 07/30/2019 (Tue) 23:59:27 Id: f7bf46 No. 13380 [Reply]

I had a mixed week. The duplicates overhaul is done, with some final duplicate filter improvements, advanced thumbnail commands, and some updated duplicates help, and I fixed a heap of bugs. The release should be as normal tomorrow.

Anonymous 07/31/2019 (Wed) 09:40:20 Id: 339ff7 No. 13382

>>13380 I'm always looking forward to duplicate filter improvements.

Version 360 hydrus_dev 07/17/2019 (Wed) 23:09:37 Id: ad39a7 No. 13223 [Reply] [Last]

https://www.youtube.com/watch?v=LQcKXieQXN8 windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v360/Hydrus.Network.360.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v360/Hydrus.Network.360.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v360/Hydrus.Network.360.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v360/Hydrus.Network.360.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v360.tar.gz I had a great week. There are a bunch of little things and an important speed overhaul to tag autocomplete. tag autocomplete I have never really liked the tag autocomplete workflow. It once blocked the UI completely, and some unusual timing options were needed to make it even useable, but it still often lagged out or just responded with gigantic lists in judders. After chipping away at the problem, this week I am finally updating it to what I really wanted.

Message too long. Click here to view full text.

35 posts and 1 image omitted.

hydrus_dev Board Owner 07/27/2019 (Sat) 18:35:46 Id: 9ecf9b No. 13323

>>13268 Hey, just fyi: the 'speculative' search and the other options are inclusive of the more narrow choices. Speculative is really just 'hamming distance <=8', so it includes the others, which are 4, 2, and 0. When I eventually add videos, I'll add gifs as well. It'll work on frames, so your pokemon gif should match enough of the still images to get linked up with the eventual dupe/alternate groups you create here, and will appear in 'speculative' searches and so on. For finding things in trash, if you change the search domain, either on the duplicate files page or a page initialised with a 'similar files' system pred, from 'my files' to 'all local files', that should include trash. You may need to be in help->advanced mode to see this option.

hydrus_dev Board Owner 07/27/2019 (Sat) 18:46:34 Id: 9ecf9b No. 13324

>>13271 >>13270 Hey, you should be able to do this from manage tags right now, although it is advanced and debug-tier UI. First, make sure help->advanced mode is on. Then select your thumbs and open manage tags. Hit the cog icon and then 'advanced operation'. It'll load the same mass-copy dialog you can do service-wide from review services, just limited to that selection. Double-check you have everything set how you want, and then click Go!. Make a backup before you get into this, as it is a powerful tool and if it goes wrong, there is no easy undo. This dialog will get a pass in the upcoming PTR/tag services work I am planning. I'd like to add the tag filter object to it so you can more finely determine which tags are moved around.

hydrus_dev Board Owner 07/27/2019 (Sat) 19:00:25 Id: 9ecf9b No. 13325

>>13285 >>13286 Yeah, the current duplicate files search is a big atomic job, and if it goes on too long with a manually started job, the UI gets caught up until it eventually finishes. I don't like it, and as I move it to this new unified maintenance pipeline, I'd love to have it work in smaller async chunks like the file maintenance stuff. Since you have a big db, I recommend you have it work in idle time (the cog icon on the duplicates page should allow this) and let it catch up in that system, which has improved cancellability. I generally recommend users start with 'exact match' (i.e. 0 hamming distance) search to start with and work that queue completely before going for larger distances. This is much faster than the other searches and gives you easy decisions first. Because of the internal logic of the duplicates system, easy/actual duplicate decisions are more powerful/useful than alternate decisions–they eliminate the queued pair count faster.

How do you tag? Metholodogy discussion thread Anonymous 05/25/2019 (Sat) 01:24:07 Id: 3afb9c No. 12692 [Reply]

To quit spamming the Q&A thread (sorry) I agree this should be made into its own thread. I'd like this to be a place to ask questions on "how should I tag this" for those that don't use the PTR and think you may be able to be more intelligent with how you tag. Also a place to discuss how you tag things and, most important, WHY that way. For example, what sort of issue were you looking to solve or what did it make easier by doing it that way? T o start off the discussion - I'm going to write a small essay on my thoughts on tags/tagging. It is my belief that tagging should be, above all else, two things 1) Simple - There are few scenarios where you must think if something should be tagged or not. Edge cases and oddities should be minimized at every opportunity. The number of tags should be kept small enough to maintain the entire list in memory. This is important for consistency. Speaking of which, the second item is… 2) Consistent - Tags should be well defined and used consistently. If a tag is often forgotten to be applied then it is a bad tag and should be either removed from the system or renamed/redefined as to be applied more consistently. In order to keep consistent tags it is also a requirement that all tags are able to be kept in memory to be applied properly. If you have so many tags that you are incapable of remembering them all then you will likely be tagging inconsistently. It is okay to add new tags to cover areas. A tag that is used infrequently because the number of items in the gallery are too few is acceptable but is discouraged if the term is rarely searched for (either as an inclusive or exclusive term). Remember: tags are meant to help you search for images. This is important when it comes to naming tags. I use a collection of 288 unnamespaced tags and 11 namespaces. The tags are mostly descriptors of clothes, physical traits, and objects. Namespaces are used for more meta information like character: artist: series: or whether a piece of art is SFW or pornographic. Some of the more heavily opinionated choices I've made in regards to my tags include: The tag "dog" is for actual animals and anthropomorphic characters are tagged using a namespace of "species:". There are no "doggirl", "catgirl", "mousegirl" tags, but "species:dog", "species:cat" instead. This rule was made because of Kemono Friends, Strike Witches. Because the rull is for all anthropomorphic characters, I end up with humorous scenarios like Upotte! and Girls Frontline using species:gun and Azur Lane using species:submarine, species:cruiser, etc. Along these lines, "cat ears" is not a tag that is used. "animal_ears" is the tag for all sort of animal ears and to specify which kind of animal ears, include a species: tag in the search.

Message too long. Click here to view full text.

13 posts and 1 image omitted.

Anonymous 07/24/2019 (Wed) 09:00:38 Id: 7ebc5e No. 13284

>>13156 >>13219 Not the other guy but what about a collaborative parent and sibling database? I mentioned this once to the dev awhile back when he first added the option to import and export parent and siblings.The idea is something where everyone can contribute to and add to their client database. For parent tags, my biggest issue is trying to connect each and every character to a series or a sub series to a parent series. If there was a parent database that already did this then this could save a lot of time for some people. A sibling database though might be a bit tricky to pull off as not everyone is going to agree on a certain term used for a tag and certain people can be pretty autistic about their tags. Over the years of using Hydrus, I've noticed how some booru and sites create useless tags that clog up my client. Some would have misspellings, other times I would have tags like "buttjob", "assjob", "butt job", "ass job", "hotdogging" or tags like character:[first name] [last name] and character:[last name] [first name] when they all mean the same thing. With a collaborative sibling database, not only could we eliminate stuff like this by merging everything that means the same but, also pick and choose which term you like best and have hydrus remember that from now on even when you download a new database. This would also be great for sites with different language tags like Pixiv. Something like this though would also mean you'd need a site other then github and 8chan to pull this off as you'd also need a place to decide and discus a tagging schema that everyone can look through and follow.

Anonymous 07/24/2019 (Wed) 14:50:45 Id: 5982e6 No. 13287

>>13284 > Not the other guy but what about a collaborative parent and sibling database? I mentioned this once to the dev awhile back when he first added the option to import and export parent and siblings. > The idea is something where everyone can contribute to and add to their client database. For parent tags, my biggest issue is trying to connect each and every character to a series or a sub series to a parent series. If there was a parent database that already did this then this could save a lot of time for some people. > A sibling database though might be a bit tricky to pull off as not everyone is going to agree on a certain term used for a tag and certain people can be pretty autistic about their tags. Over the years of using Hydrus, I've noticed how some booru and sites create useless tags that clog up my client > With a collaborative sibling database, not only could we eliminate stuff like this by merging everything that means the same but, also pick and choose which term you like best and have hydrus remember that from now on even when you download a new database. This would also be great for sites with different language tags like Pixiv I would say that the first step of solving that issue is to have a wiki and a forum/chat dedicated to sorting those issues out. We urgently need a platform to debate such issues. Here is a basic idea: tag relationship diagrams like [core word] [derivatives] [misspellings] [disambiguation] and [{foreign language}] for siblings, and [subsets] [supersets] for parent/child. From these we create a page for each tag, from that we can start and sketch out a taxonomy that makes everything clear, like how Wikipedia manage their pages. If I am allowed to pick a repo, I would choose https://github.com/Requarks/wiki for Javascript-based or maybe a Python-based wiki if the whole of Javascript is a no-go (but it definitely needs page crosslinks, chats/forums, moderation.permissions, branching/forking, archives, page versioning, page categorization/heirarchy and other needed features)

Anonymous 07/24/2019 (Wed) 14:57:42 Id: 5982e6 No. 13288

>>13287 example https://github.com/edgewall/trac https://github.com/moinwiki/moin See also https://wiki.python.org/moin/PythonWikiEngines https://www.progracoding.com/10-best-wiki-cms/

Version 359 hydrus_dev 07/10/2019 (Wed) 23:05:24 Id: 1559ea No. 13160 [Reply]

https://www.youtube.com/watch?v=3y1dO4nNaKY windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v359/Hydrus.Network.359.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v359.tar.gz I had an excellent and very full week. file maintenance The new file maintenance system now has some nice UI under database->maintenance->review scheduled file maintenance. It has two pages. It is a little advanced, but if you have some hydrus experience, please feel free to have a play around with it.

Message too long. Click here to view full text.

9 posts and 1 image omitted.

hydrus_dev Board Owner 07/15/2019 (Mon) 16:11:25 Id: 1559ea No. 13188

>>13178 I am glad you like the sound of it. The program does not phone home anywhere, but if you are thinking about using the program and are truly skeptical, I recommend you block it with your firewall to start with. It shouldn't try to make any connection unless you tell it to through a booru downloader etc… If you aren't a user yet, your enthusiasm may dim a bit when you stumble over some of the unprofessional design or buggy code, though! Everything is breaking all the time. I am a typical Anon and am trying to make the sort of software I want to use. Privacy and user control is very important to me, so it is baked into the program. I love the imageboard community and enjoy giving back to it through my work. If you do use the program, let me know how you overall find it after a few weeks, once you are comfortable with it. Feedback from new users is always useful.

hydrus_dev Board Owner 07/15/2019 (Mon) 16:13:16 Id: 1559ea No. 13189

>>13179 Great, thanks for letting me know. I'm not anywhere near ready to start work on this, but next step for the whole downloader system will be some sort of subscription + version checking system for the various scripts, like you'd see in Nexus Mod Manager, so script authors can update people more easily when sites change like this.

hydrus_dev Board Owner 07/15/2019 (Mon) 16:25:02 Id: 1559ea No. 13190

>>13181 Thanks, I will keep thinking about this. >>13187 Hmm, that is odd. When it crashes, does it throw up any error window at all, or just close itself? Does the close look 'clean', or does it just die in one frame? My first guess is the disconnection caught the db while a transaction was flushing to disk and so there is some bad data in there, although usually that stuff is recovered from. If you go to /Volumes/Drive/hydurs/db and check your latest client.log file and scroll to the bottom, is there any traceback error there for what happens after the 'booting db' phrase? A db error generally shouldn't cause a crash, so it is also odd that you aren't getting an UI popup of some sort saying 'hey, the db had a problem on boot'. If it is erroring silently and quitting the program neatly, there could be some error in there, but if it is a full-on actual crash, there won't be any kind of tidy-up statements. If there is no error, let's check your db files. Here's a help file that is a slight pain in the neck to get to as an OS X user: https://github.com/hydrusnetwork/hydrus/blob/master/db/help%20my%20db%20is%20broke.txt It will walk you through some integrity checks to make sure your db is all ok. I think there is an OS X sqlite3 executable in your App, if you know how to navigate that, or you can get the tools here: https://sqlite.org/2019/sqlite-tools-osx-x86-3290000.zip

Message too long. Click here to view full text.

Version 358 hydrus_dev 07/03/2019 (Wed) 23:38:21 Id: 5f3432 No. 13106 [Reply]

https://www.youtube.com/watch?v=s_sQcYNHTcY windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v358/Hydrus.Network.358.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v358.tar.gz I had a great week doing duplicates work and fixing bugs. duplicates I split this big duplicates storage overhaul into three jobs, and this week marks the third and final job done. Like alternates and duplicates information, potential pairs are now stored in a unified and more efficient way.

Message too long. Click here to view full text.

12 posts and 1 image omitted.

hydrus_dev Board Owner 07/13/2019 (Sat) 22:10:31 Id: 84fe6d No. 13175

>>13144 Searching for multiple 'similar files' from right-click is probably doable. Sorting them the other end is probably a hassle, but we'll leave that for later. I'm supposed to be extending system:hash to take multiples soon, so I can probably do them together. >>13145 That's a good idea. I would say hit right-click->select->(something) where 'something' would be like 'remote' files or similar and then hit remove in a second action, but I can't remember if there is a good option to select those files. I'll make sure that select menu is comprehensive and see if I can do a quick submenu that that does remove. Maybe if no files are selected, the remove menu instead branches to a similar select-type submenu. I often do 'select->archived … remove', so it would be nice to do that kind of stuff in one action.

hydrus_dev Board Owner 07/13/2019 (Sat) 22:11:23 Id: 84fe6d No. 13176

>>13149

hydrus_dev Board Owner 07/13/2019 (Sat) 22:17:29 Id: 84fe6d No. 13177

>>13152 Not precisely. I don't have the help written yet to explain this nicely, but duplicate files are now stored in a valueless group that has a single best 'king' file. In the coming weeks, I will add something like 'system:is_king'. I think if you combine something like [ 'system:num_duplicates: duplicates>0', 'system:is_not_king' ], that will let you search these non-king (i.e. worse) files while excluding the good ones, and you can then deleted them en masse. Now I have this simpler duplicates tech, I expect it to slowly appear in more locations. A nice plan is to have the main media viewer say 'hey, this file has a worse quality dupe', a little like how danbooru shows this info in a file page, and have a quick way to flick to that without interrupting the main 'list' of files the media viewer is looking at. See if the new search predicates help you do what you want here, and let me know what else would work for you.

Release Tomorrow! hydrus_dev 07/10/2019 (Wed) 01:47:43 Id: 27d0d5 No. 13153 [Reply]

I had an excellent and very full week. I have extended the new file maintenance system and written some UI to manage it, added some undo/reset functions for the new duplicate system, fixed a variety of bugs, and figured out an advanced-users-only extention to the IPFS service to permit the experimental 'nocopy' pin mode, The release may be late tomorrow, as there is more than usual to test.

Anonymous 07/10/2019 (Wed) 13:55:09 Id: 90567e No. 13154

>>13153 > and figured out an advanced-users-only extention to the IPFS service to permit the experimental 'nocopy' pin mode Please dumb it down to the masses such that IPFS can be used by any pleb. HACK THE PLANET!

Anonymous 07/10/2019 (Wed) 19:29:12 Id: 626233 No. 13157

is it possible to delete whole filename: from PTR?

hydrus_dev Board Owner 07/10/2019 (Wed) 21:31:53 Id: e5c95e No. 13158

>>13154 It'd be nice if we can figure out an easy-go mode for IPFS. We'll see how this works for the advanced users, and I expect I'll figure a read-only easy mode downloader for multihashes using http gateways, and we'll iterate off that. >>13157 Not easily yet, but once I am done with the current duplicates work, I'll be adding serverside and clientside tag filters to tag repositories so both ends can prohibit types of tags automatically.

Hydrus Scripts Anonymous 03/17/2018 (Sat) 16:53:39 Id: c54963 No. 8396 [Reply]

ITT post/discuss/jerk off to/improve scripts to use with hydrus One of my scripts was broken as shit and I just fixed, but I figured it was better to graduate from the Q&A thread and make it easier to find.

16 posts and 1 image omitted.

Anonymous 05/20/2019 (Mon) 10:00:46 Id: c3e71a No. 12631

>>12604 Thanks, this is pretty useful. Sankaku has an md5 lookup as well:


sankaku = open(fileName+"-sankaku.txt", 'w+')
sankaku.write("https://chan.sankakucomplex.com/?tags=md5%3A"+line)
sankaku.close()

Anonymous 06/20/2019 (Thu) 02:01:15 Id: 2f8c10 No. 12982

>>12604 >>12631 >>12602 I not able to get the md5 unless I open each image in the hydrus media viewer first, in which case there is no easy way to select them all and get it for all of them.

Anonymous 07/10/2019 (Wed) 14:03:10 Id: 4c0673 No. 13155

>>8396 Can someone bundle a torrent client with Hydrus+IPFS? Just found this https://unix.stackexchange.com/questions/44247/how-to-copy-directories-with-preserving-hardlinks (People in >>>/animu/81926 recommended Hard Links rather than soft links, so yeah)

Version 357 hydrus_dev 06/26/2019 (Wed) 22:35:53 Id: 1fadd4 No. 13035 [Reply]

https://www.youtube.com/watch?v=1BB6wj6RyKo windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v357/Hydrus.Network.357.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v357.tar.gz I had a great week fixing bugs and cleaning up other small jobs. shortcut and hover window fixes The recent menubar open/close detection did not work well on OS X and some versions of Linux. It misfiring broke some shortcut processing and hover window display. The good news is that as I went into that code more closely this week, I discovered several other menu, shortcut, and hover window problems that were causing long-term issues. All clients should have less janky hover window placement and behaviour, but particularly on non-Windows.

Message too long. Click here to view full text.

11 posts and 1 image omitted.

Anonymous 07/03/2019 (Wed) 16:40:19 Id: 8be780 No. 13102

e621 did an oopsie with Cloudflare!

hydrus_dev Board Owner 07/03/2019 (Wed) 17:12:28 Id: 52812c No. 13103

>>13100 Should be a fix coming in 358 later today! They switched up their API, and a user helpfully provided a new parser. Let me know if it still gives you trouble. >>13102 Can you say a bit more about the error you got here? Was it the gallery page step, the file page, or the files themselves? I just did a quick test here on my dev machine and it worked ok here. I have seen it kick in site-wide for just an hour or so, which I guess are DDoS protection or similar, but I have also recently had several reports like this, where it works ok for one user but not another. I presume it is an automatic CF-side rule kicking in, based on IP or some other profile. Any chance you are on a VPN? If so, if you switch server, does that fix it? I have a job to better detect the error page returned and giving a better 'CF problem' error and maybe 'ignored' status or something that can maybe try again several days later once I eventually have that download retry-later tech in. My guess is this problem will only increase in future.

Anonymous 07/05/2019 (Fri) 14:51:59 Id: 8be780 No. 13112

>>13103 Yeah CF got hit cause of muhChina, so it definitely is an issue for the meanwhile.

Version 356 hydrus_dev 06/19/2019 (Wed) 22:29:55 Id: 55d74e No. 12978 [Reply]

https://www.youtube.com/watch?v=9rP1wjEsbak windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v356/Hydrus.Network.356.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v356.tar.gz I had an excellent and busy week catching up after E3. Duplicate better/worse/same relationships are now stored more logically and efficiently, and a variety of bugs are fixed. Maintenance is also easier to control. duplicates The second phase of the big duplicates rewrite is now done! Rather than the old system of trying to store better/worse relationships for every possible member of a duplicate group, which is very complicated, requires many human decisions, and is not very useful in practise, this new system instead lumps all duplicate files into a single group with a single 'king' representing the best file of that group.

Message too long. Click here to view full text.

21 posts and 1 image omitted.

hydrus_dev Board Owner 06/30/2019 (Sun) 16:25:14 Id: 9825f6 No. 13080

>>13019 >>13020 >>13021 Yeah, I'm going to try to fudge something like this. I'm not totally sure on all the edge cases yet, but I will do something. Maybe matching the smallest x or y to be the same virtual zoom height, with some clever exception for portrait vs landscape pairs. and offsetting drag delta by the (0,0) point on both files. I haven't gone into this exact code, so I can't speak cleverly yet, but it is in my big job todo. Thank you for your feedback.

hydrus_dev Board Owner 06/30/2019 (Sun) 16:33:03 Id: 9825f6 No. 13081

>>13022 Yeah, hit network->downloaders->manage logins. There should be a gelb entry there by default. Add some credentials, set is 'active', and do a 'do login now' test. (it is worth doing a manual download as well there and then to test it worked for real–this stuff can be unreliable due to all sorts of factors). After then, if your login expires (I think gelb login lasts a year?), hydrus will try to login again automatically without you having to do anything. More info here, at the bottom: https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html

hydrus_dev Board Owner 06/30/2019 (Sun) 16:39:19 Id: 9825f6 No. 13082

>>13026 >>13028 If you have help->advanced mode on, then clicking 'custom action' in the dupe filter should let you select 'alternate' as the dupe type, which then gives you the dupe merge options and a choice to delete A, B, or both. If you do it from the thumbnails, you should get the dupe merge options too. I agree that having some sort of 'favourites' custom merge actions will be useful for alternate processing. I think I will do it when I get around to making a proper alternate processing workflow. I am trying to shy away from significant alternate processing in this primarily duplicates data structure overhaul, since it is a black hole of work that I could get lost in for months. I took away the 'set as not related' from thumbnails because in data terms, this duplicate type is very inefficient if not applied to files that already have potential status. I have left it to the dupe filter and 'show some random pairs' quick workflow. I am still thinking about bringing it back for thumbnails, but at the moment probably won't. It really is a 'set as false positive' rather than 'dismiss any potential info' as it has been previously intepreted. I'm still thinking about it.

Release Tomorrow! hydrus_dev 06/26/2019 (Wed) 00:18:00 Id: 9e71b7 No. 13030 [Reply]

I had a great week fixing bugs and catching up on other small jobs. Some OS X and Linux UI issues (including shortcuts and bad hover windows in the media viewer) are fixed or improved, some downloader-page flicker is reduced, the client api is moved forward a little, and the popup system now has a neat collapse/expand button to hide subscriptions and so on while they work. The release should be as normal tomorrow.

Anonymous 06/28/2019 (Fri) 03:05:16 Id: d7ac54 No. 13049

I'd like you to pay more attention to github

Version 355 hydrus_dev 06/05/2019 (Wed) 20:31:49 Id: 9c60aa No. 12861 [Reply]

https://www.youtube.com/watch?v=CFVm5R_dxoo windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v355/Hydrus.Network.355.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v355.tar.gz I had a good week catching up on small jobs. duplicates I added a 'trash file' button to the duplicate filter hover window, tucked up in a new row with the cog button so they fit on thinner windows. If both potential files are worth deleting, hitting this is an easy way to dismiss them immediately. I also fixed some over-aggressive potential pairs removal from last week's alternates update. Users with many alternates may have lost some valid intra-alternate-group potential pairs, which I expect to add back in once this whole overhaul is complete.

Message too long. Click here to view full text.

33 posts omitted.

Anonymous 06/19/2019 (Wed) 20:15:15 Id: bb23e3 No. 12976

>>12967 Thank you. I appreciate the frequent updates.

Anonymous 06/19/2019 (Wed) 22:29:07 Id: a1cd23 No. 12977

(8.12 KB 1426x41 client_2019-06-19_17-06-20.png)

>>12968 The reason that It did this was an accident. I wanted to click around it, above and below to see what the image was because this particular on had a 'prefer alternate' delete reason, so when I did that I got the second image. this is fine and all, but the issue is this is also what shows up when hydrus has a bad image/database conflict. in the case that something like this happens, because inside the import status you are able to highlight files to view, would it be possible to have a 'file deleted' hydrus icon? some text to signify its not an error on the thumbnail >>12969 With razor, unless they changed it, you shouldn't need a cloud profile to use it, but razer software is absolute fucking garbage to the point it makes logitech's look good. I would honestly look into point of service keyboards instead, as some of these can have a few hundred keys but cost 16$, there are also programmable keyboards, or what I would likely use as my second option, a teensy https://www.pjrc.com/teensy/ and crash course on learning that. >>12970 that would at the very least signify the input worked. >>12971 no, those are 294291 files that are in the db, more or less every single file that is not either a png or a jpeg, currently there are 4,010,123 files in my db, that would mean that there should be 3,715,732 files in the eligible files area, this would mean there are 1,086,053 files that are not currently seen by the duplicate process.

Message too long. Click here to view full text.

hydrus_dev Board Owner 06/22/2019 (Sat) 19:51:36 Id: 80f670 No. 13001

>>12977 Thanks for clarifying. Yeah, I'll stick the trash icon on 'remote' files that were known to once be in the client. I think thumbnails know this already, so I can just draw the icon on. My thing with these game controllers is having the side joystick. Feels so good to WASD with just my thumb, I can't see myself going back to a regular keyboard for vidya. I may bite the bullet of a Razer. I did some searching a while ago, and I think you need to 'initialise' your Razer controller with the cloud account, then you set your shortcuts, and can then set it to 'offline' mode. You can't edit the shortcuts completely locally and it is seen as a default device until you do it the first time. When my G13 finally packs up, I'll just have to bite the bullet. I'm ok with having a cloud bullshit driver if it is only ten minutes before I firewall it, even if I find it atomically stupid. I understand your count issue better. I am not sure about the million files. Normally, a handful of files will be discarded if they have no useful search data–such as completely black/single-colour images–but that does not account for so many. As I do more of the current duplicates work, I will be integrating the 'phash regen' that that maintenance stuff refers to to the new file maintenance manager. If your client has somehow forgotten or missed that big chunk of files, it will be easier at that point to requeue them for generation. I'll have some UI for it under the database menu. At that point, I recommend you do something like 'regen missing search data for: system:filetype=png/jpg'. Adding skip is a bit tricky just because it adds complexity in storage, workflow, and UI. I haven't reworked the potential data storage yet so I can't yet say cleverly if this would be ok or an utter pain in the neck. I will keep it in mind, but I suspect I will not add it.