/hydrus/ - Hydrus Network

Archive for bug reports, feature requests, and other discussion for the hydrus network.

Index Catalog Archive Bottom Refresh
Name
Options
Subject
Message

Max message length: 12000

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)

Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

US Election Thread

8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

(111.13 KB 400x400 1394958218797.jpg)

Q&A Thread: For simple questions that don't need their own thread Anonymous 07/04/2018 (Wed) 23:06:01 Id: 8ea9e7 No. 9327
Here you can ask questions so that the board is not clogged with small threads. >>6021 has reached its bump limit, so I made a new thread.
I just imported a bunch of files while having some censorship filters and that broke a lot of parents/siblings relations, is there any way to force hydrus to redo them?
Hydrus consistently crashes if I leave it processing gallery downloads while I am not using the computer for an extended period of time. It doesn't crash while doing the same lengthy tasks while I am actively using the computer. I have checked all of windows 7's advanced power settings and made sure my computer is never entering a sleep-like state or otherwise diverting power due to inactivity, but the problem persists. Anything I can try to fix this?
I'd like to use SyncThing to sync up the databases between my Windows desktop and my Ubuntu laptop. Would migrating the database to a shared folder as described (https://hydrusnetwork.github.io/hydrus/help/database_migration.html) be the best way of doing this? I don't want to unintentionally bork my files. Many thanks for the excellent software by the way, it's renewed my interest in collecting images.
Is there any way to customize the image viewer? Stuff like getting rid of the tags on the side and showing 2 images at a time.
>>12491 Hi based dev, different anon here on Linux Mint. I think this might be related to my own problem and just wanted to confirm it. When I first made the switch to Linux a few months ago, my subscriptions worked fine for the most part. There were a fair few of the files that would download, hang for a split second at 100%, and quickly say "error!" before moving on to the next file. I assumed it was due to my new place's poor internet connection, but I've noticed that Hydrus keeps trying to redownload the same files to no avail. Fast forward to today and Hydrus now goes through several dozen of these errors before moving on to newer files. The newer files are more of the same, most download fine but there are still some errors mixed in. So I'm assuming that Hydrus has been accumulating these "erroneous" links ever since I started using it on Linux. Can you confirm if this is this a known symptom of Hydrus not knowing what to do with webms after it downloads them? If it is, do you have any plans to address Hydrus' Linux webm support in the near future? Regardless, thanks for all of your hard work dude.
Is there any way for me to find zip/rar files? using filetype:zip or filetype:rar doesn't seem to work at all. Yet these types of files are visible in the database (found them by sorting by filesize > largest first).
>>12496 Unfortunately, siblings, parents, and censorship is a rats nest when it comes to system interaction at the moment. I recommend opening the files in manage tags and hitting the new 'fix siblings and parents'' button, but I can't promise anything. Like much else, it strains at the seams and is all pending a big rewrite.
>>12497 Thank you for this report. I am sorry you are having trouble. I haven't heard about consistent crashes on Win 7 before. I have two first thoughts: Could your video card drivers have some problem with doing OpenCL for client.exe? OpenCL is GPU hardware acceleration for image and video processing that hydrus uses when importing files. Your driver may have per-executable options for this–if you turn off OpenCL (may also be called CUDA for Nvidia, if you have an old card), does that improve stability? Is the program minimised when you leave it alone? I remember a long time ago, some Linux users had problems with minimised hydrus queueing up various UI events that couldn't be executed due to the program being minimised, which ultimately crashed the program. If you leave the program alone, but as a window, does that improve things? Otherwise, what happens if you turn on help->debug->report modes->db report mode and leave the client alone? It'll make lots of popups and write them to the log about current db work. When you come back and find it crashed, please then check the client log in your install_dir/db directory and scroll to the bottom–what was it doing last, before it crashed? Is there any obvious cyclic pattern or similar in the minutes before the crash? Feel free to cut the relevant data and pastebin it here or email it to me to check more closely.
>>12515 I don't recommend storing the actual db files (client.db and its friends) in a non-local folder, like a network drive. Many network interfaces do not provide the true locking ability that SQLite wants to do its work reliably. If this is your shared folder solution, you can give it a go, but I strongly recommend you make a backup somewhere safe first. I'd be interested in how it goes. If it doesn't work, or your SyncThing idea is somewhat this already, I'd say having two scripts that overwrite the db and its files from the Win and Linux machines and back again any time you want to switch over. If this is practically only a five minute job for your situation and you switch between the machines on a reliable schedule, this could work ok. If you were feeling clever, you could store the client_files folder on the shared folder and just copy the client.db etc… db files back and forth, which could take just like 30s. Otherwise, there is no problem having the Windows or Linux versions talking to the same db. As long as the relative paths line up right and only one client talks to the db at once, you should be good. Please let me know if you encounter problems. I am really glad you like it!
>>12521 Not really, I am afraid. >>12524 Yeah, the situation is bad when the client can't find ffmpeg. It generally assumes it has it. I will make a job to improve the handling here–I may even be able to put in a catch for "this client can't do video, so bail on any video downloads". Can you do me a favour and try to put one of these https://johnvansickle.com/ffmpeg/ static build ffmpeg exes in your install_dir/bin folder? Do webms then work for you? Does your Linux have ffmpeg, btw, if you just open a terminal and hit 'ffmpeg -version'? I know some users just didn't have it for some reason, and since Linux Hydrus assumes you have it rather than bundling it in the release like for Windows, that was the real problem, rather than PyInstaller path environment fun.
>>12536 Is 'filetype:zip' a tag you are searching for? My PTR has a variety of filetype and 'medium:webm'-type tags, which I am leaving in place for now but will eventually purge. Please try system:mime, and select the 'application' archive mime types you would like. Does that work?
>>12547 That did exactly what I needed it to, thank you so much!
does hydrus have a "open file location" option for images? I know they're gonna be in random clusters of fucky folders, but I need it for drag and drops since doing it from the hydrus client doesn't seem to work everywhere
>>12561 right click image >share>open>in file browser
Is there a way to do a partial Hydrus backup? Don't exactly want to make my external HD go through copying 50GB of images in one go. Would just copying a few folders over at a time work?
the rightclick menu on an image has an "archive" option, what does that do exactly?
>>12567 It sends the image from the inbox to the archive.
>>12561 >>12562 You may need help->advanced mode on for this to work, and it doesn't focus the actual file for most Linux default file managers iirc. A little funky on Windows, too, but if it doesn't select the file on the first attempt, try again. For weird drag and drops, try options->gui->BUGFIX: Discord file drag-and… checkbox. Other solution is to DnD to a staging folder, like your desktop, and then to your destination program. It is all OS-level inter-application DnD permission bullshit that I unfortunately don't have easy access to.
>>12565 Yeah, for any kind of clever backup, use a program like FreeFileSync while the client is shut down. Set up a backup of your install or db dir and do it in parts. The db/client_files folder is where all your files are. The 256 subfolders in there in the format 'fxx' are where your files are. Just do them in batches of 32/64/whatever. You can just copy manually if you like as well. More info: https://hydrusnetwork.github.io/hydrus/help/getting_started_installing.html#backing_up https://hydrusnetwork.github.io/hydrus/help/database_migration.html
>>12567 >>12568 When new files are imported to the client, they get put in an 'inbox', which is basically a special kind of tag. I designed it to work like your email. They get the little 'envelope' icon in their thumbnails. It is an easy way to mark new things that you likely haven't looked at once yet. You can find files in your inbox with the 'system:inbox' search term, and 'system:archive' to find those that are not. I recommend you set up a workflow where you process your inbox files in batches of, say, 100 at a time. Anything you like, you can hit F7 on (send to archive) to say 'keep this', and anything you do not like you can delete. Once you have played around with this a bit and feel comfortable, check out the 'archive/delete filter', which is a fast way of doing this job. More to read at the bottom of this document: https://hydrusnetwork.github.io/hydrus/help/getting_started_files.html
(1.13 KB 137x32 ClipboardImage.png)

>>12543 >have to manually select the files Time to fucking kill my CPU.
>>12545 SyncThing isn't a networked drive, it uses PTP to sync files between two local folders on different devices. It sounds like this is likely to break Hydrus, but I can give it a try. Ideally I would like to have the same database on both machines…
>>12575 If you can easily turn the sync on and off, or otherwise make it only push when you tell it to, you are good. As long as you can trigger a sync only when both clients are off, you are good. But any kind of continuous sync will not work, I think. I know a user who had his Google cloud backup sync working all the time on his hydrus folder and it ate a ton of bandwidth and messed up SQLite's write locks while it was working and actually broke things locally.
>>12573 I recommend adding a unique local tag like 'muh sibling swap' to help batch this job up. The implicit default file limit on any normal search is 10,000, which I recommend. So: Apply that tag to all files (i.e. search for [files you want to swap, '-muh sibling swap'] and then apply the tag to batches of 10k files) Search for 'muh sibling swap' in batches of 10k and: - Do a big ctrl+a, F3, swap siblings and add parents - Go to local tags and remove 'muh sibling swap' Refresh search to get another 10k. Depending on your situation, each job might be 20s or a few minutes of CPU work. If that is 22 jobs of 10K files for your inbox, then plan it over a day while you are working on another computer or something else that doesn't need a lot of CPU on the same machine and it'll be manageable. Do not try to remove the file limit and apply to all 200K images in one go. These operations are not usually linear. I'm willing to bet that sibling replace is n-squared, so a single job 20 times bigger will take 400 times as long! If 10k mega-lags for you, you can even play with adding system:limit=3300 or something, although of course you are then increasing total human overhead of the job. However, since I now have this auto-fix tech, I expect I'll duplicate it at the db-level and at some point just let you auto-apply it to everything. You might like to just wait 12 months for that to come in instead. If you give it a go, let me know how you get on!
Is it possible to add a sub tag to the system:archive to create a clear separation between types of porn. When I browse without tags, I want to be able to see only one type of porn, like in a normal folder.
>>10095 So if I change my mind about a tag name, I have to individually remove the old tag and add a new tag for every file? This is crazy pants for a program centered around tagging. What am I missing here?
>>12593 >What am I missing here? Tag siblings.
How do I force a specific white space character? I want to disallow spaces and force all tags to use underscores as spaces. eg: "short hair" should always be "short_hair". I understand I can search for each of my several hundred tags and mass update them but this is a very slow process due to the sheer amount of images that need to have tags fixed.
>>12596 >Underscores in a tag such as "Short hair" >Not "Hair: Short Hair" You disgust me.
Is there a method to save the threads that goes with the images?
>>12610 >Hair: Short Hair >not hairlength:short Step up.
>>12610 Namespaces are for meta information like the artist, name of the character, series the character is from, whether the image is lewd or not, etc. Tags are for descriptions of what the image contains. Short hair is a description of a character in the picture. With very few exceptions a namespace should only exist once per image. A notable exception is when a picture shows a character who exists in multiple series (eg. Rage of Bahamut / Granblue Fantasy or if you're someone who tags seasons as separate series) or characters from multiple series (eg. crossovers). You're doing it wrong because you end up with garbage like the following on a single image featuring two characters. >hair: long hair >hair: short hair >>12626 You're doing it better in terms of naming, but you're still doing it wrong.
>>12629 sounds like a very subjective description of namespaces to me. not sure what the difference to you is between having two namespaced tags "hair:short","hair:long" and having two regular tags "short hair","long hair". Either way you have two tags. In fact, using namespaces liberally for tags that are important to you (e.g. if you like to add lots of hair descriptors) is useful because the tags will be grouped together when displayed. When checking which such tags are already on a file, it is much easier to find them if they are grouped together. Maybe you have almost entirely files which contain only one character? I know I don't
>>12593 >>12594 Yeah, try tag siblings: https://hydrusnetwork.github.io/hydrus/help/advanced_siblings.html The system is very imperfect, but it generally works. More time will be put into this this year, including better clientside preference management for tags synced over the PTR.
>>12596 There is no solution for this yet. I expect a future version of the tag siblings system to allow some sort of regex-based replacement (think a global rule for 'replace all "\s" with "_"') that will allow this. Since desiring underscores to either go away or be mandated is common, I am likely to hardcode a solution to manage this earlier than a generalised regex solution, but the tag siblings system just isn't clever enough to handle it yet.
>>12629 >>12632 I have yet to come down to a good description for what namespaces are myself. Here is my current first draft: Namespaces are good when: They are a higher category (evangelion is a series) They are useful to highlight (characters are important) They are useful to search for (creators are often searched for, and 'creator:*anything*' has value) They are useful to group together (multiple creators is useful to know, when listed next to each other) I've been approving 'clothing:' siblings on the PTR for a little while now and I overall like it. I'm mixed on some others though. My ideal solution here is to extend tag siblings to allow clientside preferences and then for you to say "If a group of siblings includes one with 'hair:' namespace, prefer that". I have found that it is easy for users to objectively agree that 'hair:long hair' and 'long hair' have the same semantic meaning, but the big subjective disagreement is over which is better. An actually bad namespace, imo, is one that breaks the first rule above. 'male:erection' is an artifact of how some gallery sites do various female/male focus on tags, but I would rather 'male erection' (which could nicely have parents 'erection' and 'gender:male'). 'erection' is not a 'male'. Also 'erection:male'. 'general:coffee cup' and 'object:pencil' are technically correct but overspecific for most users, but again I think the specificity of namespaces is highly subjective, so the true answer is to let users define what namespaces tags could have and let them then customise what they prefer to show. If you were going for 'hairlength:short', I'd probably try to aim for hairlength:short hair for similar reasons, or go to the higher category of 'hair:short hair'. 'short hair' is both a 'hairlength' and a 'hair', but 'short' can mean a bunch of stuff. I am not confident though. NEW THREAD >>12641


Forms
Delete
Report
Quick Reply