/hydrus/ - Hydrus Network

Archive for bug reports, feature requests, and other discussion for the hydrus network.

Catalog Archive
Name
Options
Subject
Message

Max message length: 8001

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)

Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

Ghost Screen
Don't forget the global announcement this week
Saturday Evening


8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #8 Anonymous Board volunteer 09/27/2023 (Wed) 16:03:14 No. 20352 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Users can choose to download and share tags through a Public Tag Repository that now has more than 2 billion tag mappings, and advanced users may set up their own repositories just for themselves and friends. Everything is free and privacy is the first concern. Releases are available for Windows, Linux, and macOS, and it is now easy to run the program straight from source. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . Hydrus is a powerful and complicated program, and it is not for everyone. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/ Previous thread >>>/hydrus/19641
Edited last time by hydrus_dev on 09/30/2023 (Sat) 17:42:52.
771 posts and 178 images omitted.
>>21123 Apparently hydownloader does this but that shit barely works, it would be a godsend
New thread here: >>>/t/14270 I'll post v557 to it later today, and this thread should be migrated to >>>/hydrus/ soon. Thanks everyone!
>>21121 Nothing to fix in Hydrus. It's all on them. The Dev would have to do something internally with new ways of approaching the problem, and then they would just change again. I've had problems with subscriptions for Pixiv as well. Subscriptions work fine if it's just catching up with a few days. But beyond that, things seem to go weird. If you're going back further than that, you need to do a full gallery download, and just watch to see when it catches up (i.e. a lot of already in db status, showing you've caught up to where you stopped downloading the last time)

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #7 Anonymous Board volunteer 05/02/2023 (Tue) 23:18:48 No. 19641 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Users can choose to share tags through a public tag repository if they wish, or even set up their own just for themselves and friends. Everything is free and privacy is the first concern. Releases are available for Windows, Linux, and macOS, and it is now easy to run the program straight from source. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . Hydrus is a powerful and complicated program, and it is not for everyone. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/ Last thread >>>/hydrus/18976
Edited last time by hydrus_dev on 05/08/2023 (Mon) 19:57:24.
707 posts and 190 images omitted.
>>20332 Not yet. Weird characters like ()[]{}_ are collapsed into whitespace in the tag search system (so if you type 'screaming internally', '[screaming internally]' will show up as a result, and vice versa), but I plan to write a special system predicate that does an expensive 'raw' tag search on exact characters and some other variables. >>20337 The neat thing about git and 'git pull' is you update in about 3 seconds, with no worries about what you are overwriting when you extract to update. I would recommend it harder as the canonical way to do it, but for Windows users the only good option I know of that gives you 'git' on the command line is 'Git for Windows', and it has a fucking nightmare install wizard. Something like 11 choice windows, all on technical 'do you want to use the native PuTTY or your own', which normal users, and especially Windows users, don't care about. I may write a 'how to install git on Windows' guide with the sensible install choices, but perhaps there is a chocolatey solution or something that makes it one line and you are rock and roll ready to go. One thing about editing hydrus_client.sh: you might like to duplicate it to and edit 'hydrus_client-user.sh' instead. That won't be overwritten when you re-extract to update, and I have a special rule in the '.gitignore' file that means git won't ever overwrite it either, or consider it a change, if you ever do migrate to using git. >>20341 As a side thing, I'll confess that I'm shit at using git. I'm slowly learning it now I'm interfacing with it more these days, but I've been a solo dev almost my whole time programming and I hate working out branch conflicts and so on. When I was learning, SVN was the bleeding edge and I only ever used it for backup purposes. Never learned properly about forking and all that until a couple years ago. My emergency button when I really fuck up the local store is usually just to delete the whole thing and checkout again. >>20345 >>20348 Based longpost. I won't add much, but if you want internal hydrus support for a site that doesn't come by default, including many of the booru.org ones, the following is a general good pattern, but bear in mind that this can get advanced, so don't worry if you would rather get more comfortable with the program first. Also, my downloader engine is really shit to update at times, and often the updating-a-broken-downloader solution is 'delete all the objects you imported before and then reimport the new ones', or you can end up with a mess of 'downloader objects' that gets in a bit of a tangle: - Go here, https://github.com/CuddleBear92/Hydrus-Presets-and-Scripts/tree/master/Downloaders , search for the site you want. - Download the raw 'png' file.

Message too long. Click here to view full text.

New thread here >>>/t/13212 I'll post v545 to it later today, and this thread should be migrated to /hydrus/ soon. Thanks everyone!
(188.18 KB 1578x1979 1705355331545551125_1.jpg)

>>20344 >I guess that's some kind of safety feature? Since git is a version control software it intentionally makes it hard to lose "progress" if you stage and commit often, so that's why it doesn't let you overwrite changes easily. It would be poor design to easily/silently let you clobber files. >commit/rebase In git, a "commit" is basically a snapshot of the files at a certain moment in time with a message "yes, these changes are real" but you don't actually need to commit, you can delete the commit before pushing, but even if you delete only the files and not the commit, if they've been commited they're still safe. Pushing a commit to the remote (server) tells everyone else that the changes are real, you can easily remove a commit before you push it but if you try to remove it afterward pushing it it's a big hassle, since it's playing with "time". A 'rebase' is what you'd do if you had diverging branches, say for instance you're working with someone else, you both pull from the latest commit, but you commit & push your work before the other person, since git is a sort of 'timeline' all commits come from every commit before, but you both started from the same point in time and now the 'timeline' is wonky, you'd have to rebase your changes onto the new branch. A rebase is a way to "easily" do that, but many find it confusing, I don't have a very good grasp of it either. You probably won't need to rebase if you're just pulling updates from a software repo, but if you're doing development work to submit changes upstream it may be necessary. >stash 'git stash push' adds all uncommited changes to a sort of side place, the stash, useful if you have important work but it's not done/you don't want to commit it yet. 'git stash pop' returns the files from a stash, and deletes the stash (just the one you popped) undoing a 'git stash push'. Think of the stash as a "fake commit" that is technically temporary. You can also easily delete it with a 'git stash drop stash@\{#}' where # is the stash number you want to delete without saving >roll back As for rolling back an update, say for instance hydrus v534 was bad, you found out before you started the database and upgraded to v534, to "roll back" to 533 you would need to use "git checkout", it can be used for branches as well as commits, so to checkout branch "test" you'd use 'git checkout test'. To checkout a specific commit there's a couple things you can do, you could use the reference, an exclusive sha-1 hash of a certain commit, Hydrus v533's first release has reference '8b3ae4ac1a006e955addefb7a8af47a4e2b6336c' so you'd 'git checkout 8b3ae4ac1a006e955addefb7a8af47a4e2b6336c '. You can also use the shortref, a truncated version of the hash for that commit, which would look like 'git checkout 8b3ae4ac'. If the commit is more recent, say it was the second to last you could go off of the HEAD position (the most recent), so you could 'git checkout HEAD^^' for 2 commits up. You can also use 'HEAD~#', where # is the number of commits up, so you wanted to go 11 commits back instead of typing 11 carets you'd just 'git checkout HEAD~11' You can /also/ use the tag, if a developer uses them, hydrus thankfully does so all you'd need to do is a 'git checkout v533' which is actually a different commit since v533's initial release had a hotfix for a type check. >In this case all I'm "merging" on my end is the chmod +x I set on those 1 or 2 files, right? If there's no other files listed when you try to pull the latest update, yes. You can run 'git status' to see what files have changed. This kind of turned into a git seminar holy hell, I'll wrap this up. If you want to read more about git you can check out the pro git book, it was helpful when I was trying to set up a git repo on my server. https://git-scm.com/book/en/v2

Message too long. Click here to view full text.


(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #6 Anonymous Board volunteer 12/21/2022 (Wed) 19:28:08 No. 18976 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Users can choose to share tags through a public tag repository if they wish, or even set up their own just for themselves and friends. Everything is free and privacy is the first concern. Releases are available for Windows, Linux, and macOS, and it is now easy to run the program straight from source. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . Hydrus is a powerful and complicated program, and it is not for everyone. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
661 posts and 142 images omitted.
>>19633 >>19635 Actually nevermind, I'm back Obviously i use the system packages which are on python3-mpv-1.0.1. On another note, could it be an issue that i build with Qt5? I didn't get PyQt6 running on my system yet, dunno if Qt5 is still supported officially?
>>19635 >>19638 Aha, I see. Sounds like you are running off your system python. I used to do this myself, but I learned about venvs and now I strongly encourage everyone to move to them for anything more advanced than simple scripting. Forgive me if you know about this all already, but venvs (virtual environments) are basically a python-in-a-box that you can install anything to without messing up your system python with new versions or anything. If I haven't pointed you at my newer 'running from source' help, please check it out here: https://hydrusnetwork.github.io/hydrus/running_from_source.html A few months ago, I wrote some really easy 'setup_venv' scripts that are in the base install of the source extract or github repo clone that set everything up for you. Please give them a go, they'll ask some questions and get all the right versions for you and set up Qt6 and all that (assuming your OS isn't super old and can't run it!) all in a few minutes. If you would rather do things more manually, check the second half of the help page, where I walk you through doing it yourself and talk about venvs more. Qt5 could be the cause of some of the jank here, and Qt6 has a whole load of improvements, so if you can run it, I recommend it. Let me know how you get on! Qt5 is supported, but I can't promise anything. Win 7 users have to use it. >>19634 >Does hydrus give a way to know what subscription an image was downloaded from? Not yet, but this is increasingly on my mind. I need to do a whole bunch of subscription-infrastructure improvement, but I do really want greater integration of subscription-knowledge into the general workflow. I want you to be able to right-click on a file and see that you downloaded it from your danbooru artists sub.
New thread here >>>/t/12094 I'll post v526 to it tomorrow, and this thread should be migrated to /hydrus/ soon. Thanks everyone!

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #5 Anonymous Board volunteer 09/14/2022 (Wed) 22:07:14 No. 18264 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
708 posts and 156 images omitted.
>>18953 >>18954 Sorry for the trouble here, it sounds like I caused some of this. I've been doing some API reshaping recently--basically I made some dumb decisions when I first made the API, and now cleaning it up and obsoleting the old way. I also messed up how I did that obsolescence in a couple cases, doing it too fast with too little warning. The main change has been in how I present tags and services. I'm moving towards the 'service_key' being the primary API id for services, and tags are now showing both storage and display contexts. I'm trying hard to move to data structures that we can expand in future without breaking anything else, so I'm hoping there won't be much more of this, and we'll have a lot more backwards compatibility going forward. I have a tight work loop and regularly make a lot of changes, which is fine when I am only devving my own stuff, but I need to be more careful with the systems used by third parties like the API. I need to do one more change in the new year, on the '/get_services' call, and I hope that'll mostly be it. If it helps, you can have the Client API log its requests under services->manage services. It gets spammy in normal usage, but is useful for quick debugging. I don't know the animeboxes guy(s), but I am happy to help if they or you have any technical questions. >>18956 >>18959 Yeah, I think the emoji stuff is based on the font your OS supplies. There may be additional Qt issues there (I remember back in the wx days, there was different emoji support depending on whether I was drawing in software or hardware, so I bet Qt has some similar issues), but afaik Qt says 'hey, how do I show this unicode?', and your OS reponds with some kind of svg instructions. 🍆 shows ok here on Win 10, as does ethiopic ፕዘጎ and cuniform 𒊕, with I'm pretty sure default fonts. My guess is this support will generally increase as OSes roll out newer font tech generally. There may be a way to improve clever font support by specifically selecting a different font in QSS, but what to actually choose is beyond my expertise. As for IME, that sucks, sorry! I'll keep my eyes open in case there are flags I can ever set to make this work better, but ideally Qt just updates one day and it all starts working. Actually, now I think of it, if you decide to run from source and PySide6 doesn't do IME, maybe PyQt6 will. They talk to the C++ layer in different ways, despite doing ostensibly the same job. Let me know if you discover anything!
>>18963 >>18964 Yeah, I'm afraid there are two things going on here: - Subscription queries 'compact' themselves down to 250 items as long as the 251st item is 'done' and old (twice the death velocity age). This is because subscriptions are a slightly weird bloaty object in order to be smarter and I have to keep them lean so they load and save fast. - Pixiv have an unusual API that gives hundreds of results in one request. The hydrus downloader engine has logic tuned for boorus, where you'll get 20-75 per page. I have some hacky pixiv-specific hardcoded subscription logic that takes the first 200 from a giganto-page like that in order to avoid other logic getting confused about how every new subscription check suddenly found 800 new files. This isn't helped by the fact that pixiv post pages can have multiple files, which frustrates some of my recheck logic. The 'quality info button' functionality is limited by the 200/250 file compaction. I think the real answer here is for me to write a proper 'this file was downloaded by x sub' metadata type and consult that, rather than the sub query's semi-ephemeral file log. I'm also thinking about expanding the statistics I offer and generalising the search domain to any file search. Then you'd be able to look up how 'good' a creator: tag was, or system:was downloaded by x sub. >>18967 I'm just a normal Anon in the lucky position to put in the work, but I appreciate the praise. I'm a little older, been around these parts since the start, which I think has helped callous my soul a little, but I'm also grateful that the hydrus feedback has been typically very kind, given how we often go about things. I try not to bullshit people or drop drama bombs, and I really do care about and want to add tools for the community, which I think has helped in that regard. I'll also say there's a certain momentum to it--you start a routine and before you know it, it feels normal. This is just what I do now. Just keep on pushing that boulder up the hill. >>18971 Not right now. I was quite opposed to this for a while, but the Client API guys convinced me to expose file ids there, so I've been drip-feeding it to the UI too. I expect you've seen you can copy the file id from the thumbnail menu if you are in advanced mode. I could add a hash_id type to 'system:hash', I expect.

Message too long. Click here to view full text.

New thread here >>>/t/11075 I'll post v511 to it later today, and this thread should be migrated to /hydrus/ soon. Thanks everyone!

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #4 Anonymous Board volunteer 04/16/2022 (Sat) 17:14:57 No. 17601 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
659 posts and 101 images omitted.
>>18216 I haven't done the big change on the Linux side just, so this may be placebo, but I've also done some other video/mpv work recently that may have reduced some lag for you. As I asked in the v499 changelog, some Linux users tested out the new python-mpv wrapper, and they said it works well, so I will be updating it in the Linux build for v500. I think you'll still see 1.109 for now, but when libmpv1 either in my build or your system (through apt-get) gets updated, the new python-mpv should handle it well. >>18217 Damn, thank you for this report. I am sorry for the trouble and will check this out. I did work on that loop-checking logic recently, I thought to improve things and help bunch stuff together for the janitors, so perhaps there is a new bug in there. >>18218 You know I was reading this recently https://support.nordvpn.com/General-info/Features/1845333902/What-is-Meshnet.htm , which as I understand it is basically user-friendly hamachi. If the other vpn providers soon offer the same tech, and consumers get access to easy virtual local networks, some of our hydrus-specific pains is the ass may evaporate. All the shit about setting up proxy forwards and self-signed ssl certificates and things disappears if the secure network side of things is handled by a vpn provider and your phone thinks it is talking to 192.168.x.x. That said, if you are already on the same network, your best bet is just to boot up the Client API and try out Hydrus Web. If you already have 192.168.x.x:45869 visible and just want to sit on the couch with your tablet and browse things, this is possible right now without too much strife: https://hydrusnetwork.github.io/hydrus/client_api.html >>18219 Probably early next year. I'm going to start with the very easy case of jpeg+png exact pixel dupes (keep the jpeg), just to get a skeleton of an auto-dupe-filter started, and then flesh it out with cleverer rules from there. Everything will be completely optional, default off. My dream is that in five years we have all sorts of tech making weighted decisions about dupes, and 95% of stuff is handled without you ever seeing it, including immediately on import. >>18220 Maybe. We've played with this tech before. A legacy system called 'file lookup scripts' allows you to do this on a per-file basis in manage tags, but I think you have to turn it on under options->tag suggestions. If I do it, it would hook into the slow-burn system I describe here >>18258 since the same bandwidth and workflow concerns apply.

Message too long. Click here to view full text.

>>18221 >>18222 I am very sorry, I think you have overwritten your database, yes. You still have your actual media, the jpegs and so on, but your archive record, known urls, the session and subscriptions, all the stuff you see in the UI, has probably been overwritten. You probably already have, but I recommend you have a very good think about if there is any chance an old copy of your database might sit in your recycle bin or an old usb stick somewhere. If there is one, we can probably stitch a rollback together. If not, you may be looking at starting a new client and then importing your old install_dir/db/client_files structure. You would be looking at the 256 'fxx' subdirectories in that folder. For the backup situation, I am sorry. I know how it feels to lose your stuff. Although it is a shame, it is usually emotional kicks like this that ultimately give you the motivation to figure out a good backup solution in future. You don't want to lose shit like this again, so you put the time and money in in future. You don't have to do it today, but if you have a planner, make sure in a week or two that you come back and plan a proper backup routine. Not just for hydrus, but your documents and writing, any art or programming you do, anything digital, so if your machine blows up or your place burns down you still have a copy on a USB stick in your backpack, whatever works for you. My backup help is here: https://hydrusnetwork.github.io/hydrus/getting_started_installing.html#backing_up And a special message for you is here: https://hydrusnetwork.github.io/hydrus/after_disaster.html I'm happy to help with recovery or setting up your future backup, just let me know. >>18225 Thank you for this report. I will investigate this. Unfortunately, crashes (as in the program halts and exits immediately) will not record any information in the log, but if the program simply halts and does not respond, there may be something in the log as >>18236 says. As you are a Win 7 user and Qt6 will not work for you, you may be looking at the option of running from source soon. The users in this chain >>18213 found that running from source reduced their crashes significantly, so if your problem is similar, this may be something to think about.

Message too long. Click here to view full text.

New General here: >>>/t/9763 This one should be migrated to /hydrus/ soon. Thanks everyone!

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #3 Anonymous Board volunteer 12/01/2021 (Wed) 23:12:07 No. 16965 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
632 posts and 156 images omitted.
https://www.youtube.com/watch?v=R1t6iNG28zI windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v480/Hydrus.Network.480.-.Linux.-.Executable.tar.gz I had a good week. Notes now display on the media viewer. notes Notes have always been a slightly hidden system, a bit like ratings were. Today is a step forward to exposing them. Any file that has notes (you can start adding notes to a file by hitting manage->notes on their right-click menu) will now show them in the media viewer, just below the top-right hover window. They get their own hover window too, if you mouse over them. If you click on a particular note, the 'edit notes' dialog opens on it. This is a first version, and a little ugly, but I'm happy we now have something I can iterate on in future. If you are a big notes person, please let me know how it works best and worst for you. If you have unusual font style, size, or colour, let me know if it goes crazy or sizes too short or tall.

Message too long. Click here to view full text.

(608.93 KB 1366x768 Screenshot_20220406_182040.png)

(765.98 KB 1366x768 Screenshot_20220406_182326.png)

(516.64 KB 1366x768 Screenshot_20220406_182538.png)

(89.35 KB 1023x1023 clapping.gif)

>>17598 Marvelous!!! It is a nice touch that when more than one note is present and clicking on that note, the Note dialog will open showing that precise tab on foreground. Something that at least for me is annoying is that Notes will open with the scroll at the end and not at the beginning as should be expected. See pic 3.
Hey, unfortunately 8chan has had some posting trouble and we have lost a week or so of posts. Since this thread is bumplocked, I was going to make a new thread for 481 anyway, so Hydrus General #4 is here: >>>/t/8151 This thread will be moved to the /hydrus/ archive soon. Thanks everyone!

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #2 Anonymous Board volunteer 04/20/2021 (Tue) 22:48:35 No. 15850 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am the hydrus developer. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. Past hydrus imageboard discussion, and these generals as they hit the post limit, are being archived at >>>/hydrus/ . If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
1111 posts and 210 images omitted.
Out of curiosity, is it possible to browse through all the tags present in the public tag repository? For instance finding out how many instances of tag A exists and perhaps how many files with tag A also has tag B
I had trouble posting this last night. I hadn't realised how long the thread was--I will make a new general today for the release post! I had a good week. I managed to add support for embedded ICCs, which will improve some images' colours, and overhauled how files are deleted from the client and server so the system is smoother and more reliable. The release should be as normal tomorrow.
>>16962 Unfortunately the sheer number of tags--tens of millions--make it difficult to 'browse' in UI, so your best shot for now is to access the SQLite database files manually and run your own statistical queries. I can help you with that if you like. Another simple option is just to run some basic searches on 'all known files'. This is advanced, so be careful of running some super CPU heavy searches, but turn on help->advanced mode, and then change your search page from 'my files' to 'all known files' and the tag domain from 'all known tags' to 'PTR'. Then do some searches, you'll see real tag counts and get ghost results of non-local files if you run a search. It is borked developer-mode stuff mostly for mine or the janitors' use, but you can learn some interesting things. New Thread >>6469

(4.11 KB 300x100 simplebanner.png)

Hydrus Network General #1 Anonymous Board volunteer 01/20/2021 (Wed) 04:21:14 No. 15143 [Reply] [Last]
This is a thread for releases, bug reports, and other discussion for the hydrus network software. The hydrus network client is an application written for Anon and other internet-fluent media nerds who have large image/swf/webm collections. It browses with tags instead of folders, a little like a booru on your desktop. Advanced users can share tags and files anonymously through custom servers that any user may run. Everything is free, privacy is the first concern, and the source code is included with the release. Releases are available for Windows, Linux, and macOS. I am continually working on the software and try to put out a new release every Wednesday by 8pm EST. If you would like to learn more, please check out the extensive help and getting started guide here: https://hydrusnetwork.github.io/hydrus/
703 posts and 145 images omitted.
>>15842 I usually do everything using bash, so I have this script I named hydrupdate: #!/usr/bin/env bash test -z "$1" && exit INSTALLDIR='/your/install/dir/here/' test ! -d "$INSTALLDIR" && echo 'The provided install dir does not exist or is not a directory' && exit tar xf "$1" -C "$INSTALLDIR" --strip-components=1 I can then call the script with hydrupdate /path/to/Hydrus.Network.<release>.-.Linux.-.Executable.tar.gz.
New thread for v436 >>3626 !
>>15839 Thank you, I will! These questions come up enough, and I sometimes forget the details myself, that there should be an official write-up somewhere. >>15840 The other guys are also correct here, but if you are using one of the 'extract' releases, basically you just drag and drop the contents of the archive onto your install directory. Longer explanation here: https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#updating >>15843 Thanks, I will have a look. I guess the third click is getting sent by Qt as a second doubleclick event. I'll see if I should be terminating the click-chain or whatever it is called when I eat the initial double-click. >>15846 Yep, check options->sort-collect. The UI is debug-tier bullshit, I apologise, but if you add some new sort types, any namespaces in those sorts will be added in your collect dropdown.

(1.59 MB 847x860 1425150841112.gif)

Remote backups with Hydrus Anonymous 01/20/2020 (Mon) 13:27:14 Id: c2f342 No. 13543 [Reply]
Hello everybody, I've been wondering how some of other Hydrus Network uses handle backups for their files. My current setup is like thus: I make weekly local backups of my database, which are then sent to a NAS through UrBackup, From there, I've been using Duplicati with cheap S3-compatible remote object storage. Now, supposedly Duplicati performs deduplication before uploading, however it still seems like it takes its sweet time on the SQLite database files themselves, so I'm wondering if their format just doesn't lend itself to be deduplicated adequately or if there is some other issue. I don't use password protection (although from what I've seen, it doesn't actually encrypt anything by itself so it shouldn't make much of a difference either way) So, what other methods do you use to backup your Hydrus files remotely? Ideally it should be with deduplication because although space is not at a premium for me, my upload speeds are very limited.
4 posts and 1 image omitted.
>what other methods do you use to backup your Hydrus files remotely? run a hydrus client on a server and set it as a remote file repository, then you can tell your local instance to upload to it.
>>15078 I'm the anon from the OP, the main issue with this approach is that without deduplication of some kind, you'll end up having to send your (rather sizable) database file with every update all over again every time you rsync with the remote folder. As I mentioned last year, storage space isn't so much a concern as it is upload speed, as even with a mild recent improvement they still top around 2Mbps. I had to stop using Duplicati as it was incredibly fragile, I'm now using borg and rclone to send my local repo to a Backblaze B2 bucket (even cheaper than my previous option). Borg handles the deduplication and encryption.
>>15120 With something like borgbackup/rsync you can do it in a way that it only updates changed files I believe. So you're not backing up the entire db over and over, only changes.

Bugs Thread hydrus_dev 02/06/2019 (Wed) 01:35:09 Id: 2b149c No. 11542 [Reply] [Last]
BUGS THREAD
500 posts and 1 image omitted.
This thread is now archived at https://archive.is/cq5Tc . I will not post here any more, as the board will be deleted next Wednesday. Please move to the Endchan thread at https://endchan.org/hydrus/res/9.html , thank you! This plan has changed. The main imageboard location for hydrus is now a General on >>>/t/ here on 8chan.moe.
Edited last time by hydrus_dev on 01/20/2021 (Wed) 04:40:11.
>>14683 >If you do have it, is MPV the player under options->media? If your hydrus does not have libmpv access, or is failing to import it, it will fall back to my native viewer, which has no audio support. Fixed. Thanks a lot. I'm new to Hydrus. Using Linux MX. Before I was using Tagspaces which is a sluggish piece of trash. Looking for a more suitable alternative I bumped into Hydrus; so far I'm very pleased by its performance and rich settings.
>>15118 Is there a way to get the blacklist back to working? There's usually a cookie for you blacklist on gelbooru but for some reason the downloader doesn't use it and stopped working for me a month or so ago. I can't remember if it was something I did or if the site changed or what.

(4.66 MB 4000x2715 shutterstock_89245327.jpg)

Parsing scripts Anonymous 11/14/2016 (Mon) 18:14:13 Id: f047d8 No. 4475 [Reply] [Last]
How about a thread for discussing/creating/sharing parsing scripts? I made one for md5 lookup on e621.net (actually I just modified Hydrus_dev's danbooru script). Let me know if I did anything wrong with it, I'm pretty clueless… but it seems to work fine.
[32, "e621 md5", 1, ["http://e621.net/post/show", 0, 1, 1, "md5", {}, [[30, 1, ["we got sent back to main gallery page -- title test", 8, [27, 1, [[["head", {}, 0], ["title", {}, 0]], null]], [true, true, "Image List"]]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-general"}, null], ["a", {}, 1]], null]], ""]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-copyright"}, null], ["a", {}, 1]], null]], "series"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-artist"}, null], ["a", {}, 1]], null]], "creator"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-character"}, null], ["a", {}, 1]], null]], "character"]], [30, 1, ["", 0, [27, 1, [[["li", {"class": "tag-type-species"}, null], ["a", {}, 1]], null]], "species"]], [30, 1, ["we got sent back to main gallery page -- page links exist", 8, [27, 1, [[["div", {}, null]], "class"]], [true, true, "pagination"]]]]]]
86 posts and 1 image omitted.
I'm not sure if GUGs can make these, but anyone have a module for setting up Youtube subscriptions?
>>14782 They're not hiding lolis. I don't understand why I keep hearing this. Did you check the mature content option in settings and clear your account blacklist? Do you have an account in the first place?
Can someone help me understand what parsing scripts are for, and how to use them? Are they to improve the amount of tags that are found for images? Like a reverse search?

Q&A Thread: For simple questions that don't need their own thread Anonymous 09/12/2020 (Sat) 23:55:46 Id: ff6d52 No. 14731 [Reply] [Last]
Here you can ask questions so that the board is not clogged with small threads. Old thread >>12641
158 posts and 1 image omitted.
>>15074 Any site that doesn't operate like a normal booru is tricky. Sometimes you just have to babysit them. I assume you are downloading through nitter, right? I think the twitter API that nitter hooks into does not deliver tweets beyond about 3,000 tweets into the past, so if you are hitting that limit somehow, or if nitter has its own limit, that may explain it. I have been getting reports recently that nitter has been giving '429 - server rources limited' or similar recently as well. This may be related, I am not sure. >>15088 I am not sure if the existing pixiv login script works right now, it has always been temperamental. If not, I think you'll have to use Hydrus Companion or similar to log in by copying your browser cookies to hydrus. Then pixiv will think your hydrus is you and it'll have access to private stuff. >>15095 Support here is not yet great. If you are importing from hard drive, they should import in filename order. If you are importing from a site, it will come in the order the site's parser pulls the URLs, which is usually good, sometimes reversed. The main problem is hydrus currently has integer timestamp for import time, so if two files are imported in the same second, they will sort randomly when you sort by import time. If this is what is hitting you, I apologise. At some point there will be a conversion to sub-second times, although a retroactive fix would likely be impossible. I am hoping the eventual file alternates work will provide a nicer system than page tags. I agree they are clunky.
>>15115 >>15116 This is some developer bullshit unfortunately. You don't need mpv, but 'libmpv1'. You should be able to get it with apt.
This thread is now archived at https://archive.is/0WoQk . I will not post here any more, as the board will be deleted next Wednesday. Please move to the Endchan thread at https://endchan.org/hydrus/res/10.html , thank you! This plan has changed. The main imageboard location for hydrus is now a General on >>>/t/ here on 8chan.moe.
Edited last time by hydrus_dev on 01/20/2021 (Wed) 04:36:35.

(32.04 KB 480x360 r1nn-tp26KE.jpg)

Version 425 Anonymous 01/13/2021 (Wed) 22:34:36 Id: 6d169b No. 15109 [Reply]
https://www.youtube.com/watch?v=r1nn-tp26KE windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v425/Hydrus.Network.425.-.Linux.-.Executable.tar.gz I had a good week. I optimised and fixed several core systems. faster I messed up last week with one autocomplete query, and as a result, when searching the PTR in 'all known files', which typically happens in the 'manage tags' dialog, all queries had 2-6 seconds lag! I figured out what went wrong, and now autocomplete should be working fast everywhere. My test situation went from 2.5 seconds to 58ms! Sorry for the trouble here, this was driving me nuts as well. I also worked on tag processing. Thank you to the users who have sent in profiles and other info since the display cache came in. A great deal of overhead and inefficient is reduced, so tag processing should be faster for almost all situations.

Message too long. Click here to view full text.

1 post omitted.
>>15109 Have you considered 9chan?
Maybe its time we moved to a generic forum. I hate the idea but we've been jumping from 1 chan to the next so quick lately. I wonder if they make accountless forum.
>>15111 They sent a message to all BOs about it, and there are some (I think new) little text banners here and there saying the TOR link on the main page and so on. My guess is relations with their host aren't super great, and with all the other deplatforming going on, they are just waiting for the email now. Sounds like the situation sucks. I don't browse here any more either other than /hydrus/, and I should have moved a while ago, I was just putting it off. Although, funnily enough, having said I would 'lock' the board, I then couldn't find that option in the admin panel. I'll have another look today once I am caught up with the main threads, but if you just can't do that, I'll make a sticky with an announced deletion date. Making an archive of the Q&A thread is probably worthwhile. >>15114 I am not sure. I considered trying for 8chan.moe or somewhere in the webring, since that is where I browse these days personally, and they are opening up board creation in more places. But I think I'll just sit at endchan for now. It is a nice quiet corner. I have never been a comfortable board owner, and never good about running and checking a board for spam etc&hellip;, so I'd probably be happier just being a janny on a board run by others, which is how the hydrus discord works, although I wouldn't want to ask people to do hotpocket work unless they are keen. >>15121 Yeah, jumping around has been frustrating. My feeling is we will get another wave of hosts setting new rules over the next 1-3 months and taking out wrongthink and then things will settle again. If you are comfortable in a more normie-friendly experience, the discord is at https://discord.gg/wPHPCUZ The plan to move to Endchan as primary board has changed. Codexx here on 8chan.moe kindly offered to host me on /t/, so I will now be maintaining a Hydrus Network General thread there.
Edited last time by hydrus_dev on 01/20/2021 (Wed) 04:44:21.

Yiff.party is Dead... Any plans to help rip patreon or SubscribeStar? Anonymous 11/15/2020 (Sun) 19:49:17 Id: 0dfb73 No. 14929 [Reply]
Kemono.party is not working right now and seems to be very unstable. A good idea would be to get patreon/subscribestar downloaders working. Currently hydrus will try to look at patreon pages then get booted out, a possible code reference for how to rip it might be here https://github.com/OpenYiff/Kemono but this is for running a kemono.party service. Any ideas? I got used to using yiff.party as a backup/3rd party ripper to get HD content to hydrus, then deduplicate the shit booru quality versions.
9 posts omitted.
>>15022 It was broken for a while but it seems to be working now. >>15020 The site has changed the URL schemes for posts since that was put on Cuddlebear's repo. I would wait until the site is more stable and/or the API is finished before trying to find/make a downloader.
>>15024 I see, that makes sense. Thanks.
I made that WIP kemono parser, its specifically for kemono.party/files and was made when the actual kemono frontend was non-functional. It was meant to do the bare minimum of scraping their bulk file CDN but I don't even think they use the same URLs for file downloads anymore.

(14.63 KB 480x360 YnU_j_ZA-tc.jpg)

Version 424 Anonymous 01/07/2021 (Thu) 01:59:59 Id: 0070e5 No. 15072 [Reply]
https://www.youtube.com/watch?v=YnU_j_ZA-tc windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Windows.-.Installer.exe macOS app: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.macOS.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v424/Hydrus.Network.424.-.Linux.-.Executable.tar.gz I had a good week. There are some quality of life improvements and faster tag search across the board. The update will take some time this week to update a cache. If you do not sync to the PTR, it will be just a few seconds. If you sync to the PTR, expect about 5-15 minutes. faster tag search In the second half of 2020, I tried several times to tune the database for different sorts of wildcard tag search, which is used in all autocomplete lookups and many file searches. I was sometimes able to get small clients always running well, or complicated large systems running, well, but I failed to get it good for all situations with code alone–the structure of the database tag lookup cache made the tuning difficult.

Message too long. Click here to view full text.

4 posts and 1 image omitted.
>>15090 Yes I agree, I will have some sort of client-to-client transfer as part of this, so users who have been using multiple clients will be able to merge neatly and preserve time and archive metadata. Around that time I'll likely have to introduce rating export/import too, and perhaps URLs as well. The good news is now we have the Client API, this should be neatly possible as a direct client-to-client connection.
>>15092 Thank you.
I had a good week. I was not able to fit much interesting fun stuff in, but I fixed the slow tag autocomplete search in the manage tags dialog, sped up tag processing, fixed some 'ghost' tag bugs, and reduced some wasted CPU in the network engine. Overall, the client should be a bit neater and faster in 425. The release should be as normal tomorrow.

(94.10 KB 1024x768 1601788051705.jpg)

/big data general/ Anonymous 10/05/2020 (Mon) 05:17:17 Id: ecf9e3 No. 14802 [Reply]
or, "i've got xxxxxxx thousand images in my inbox, what the fuck do i do?" i'm currently having this problem myself, so i thought it would be nice to have a thread to collect information about various ways to automate sorting/tagging/etc, such as: * image deduplication scripts: >https://github.com/knjcode/imgdupes i'm presently also writing my own as well which should work somewhat better for large datasets, i'll post it here when it's usable * datasets, useful for running against aforementioned dedupe scripts >danbooru2019, contains all danbooru pictures + metadata up to early 2019: >https://www.gwern.net/Danbooru2019 i remember there being large dumps of other booru metadata on here years ago, but i can't find them anymore * AI/neural network software for automated tagging, classifiers and etc: https://github.com/KichangKim/DeepDanbooru https://github.com/imamar94/ramrem-classifier things i couldn't find but would find extremely useful: >AI anime/photograph classifier

Message too long. Click here to view full text.

4 posts and 1 image omitted.
>>15084 Is that not what it does? Unless you mean applying a non-system tag like "video with audio" or something, it does do that automatically.
>>15099 Just tried adding a new url with a bunch of videos to the watch, some have audio some don't. None of them have been autotagged with any tags, except for the few that have a hash matched. So it does not seem as if it is auto-applying tags for stuff that are videos, and stuff that has audio.
>>15101 do a search for "system:has audio" you absolute retard

[ 123456789101112131415 ]
Forms
Delete
Report