>>21090 I think they have a lot more than Sankaku, and pretty much anything goes there, so you'll be sure to find whatever you're into.

>>21091 pro tip - You'll want to look at the japanese tags that are suggested when you type in the romanji name. There's usually more content under the kanji tags.

>>21089 I don't think you can import them, you will need to manually migrate them unfortunately. Yes, hydl remembers past downloads, and if you pair it with Hydrus Companion which I suggest using anyways, you can even see files you've already downloaded in your browser, see pic related, the blue border indicates "downloaded but not imported".

I've been downloading a lot of heavy files (videos) off sankaku since october and the old links in my gallery queue still work. Can't something be done to generate these old type links from new, volatile ones, before enqueueing downloads?

>>21067 The default tag changed from "wd-hydrus-tagger ai generated tags" to "wd v1.4 vit v2 tagger ai generated tags". Was this intentional? Can this default tag be managed?

Who are the russophobic liberal hypocrites running the ptr.hydrus.network and limiting the access to it by country? They dare to use our Nginx software and sperg about internet freedom and le big brother while doing exactly the same.

>>21093 Either you need high level autism or I'm missing something with hydownloader I installed it, got the GUI thing, tis all green, I add subscriptions (unpaused) and it does nothing ?????

>>20884 >What features would you like most Sorry for the late reply, thought I lost my whole database but found a backup in the dumbest place after a few weeks of giving up. Have multiple backups! It would be cool if we could do booru wiki style entries where you could link to other tags or even URLs, but to me the most important part would be ease of looking up a basic definition for that tag. Maybe type it in to the add tag box and right click a candidate tag and hit 'see definition' or something. >>20912 >why would you manually add tags when other people have done that work for you? I don't use booru tags outside of basic stuff like copyright and characters because of how unreliable and varied booru tags are. Even characters are commonly fucked because different sites disagree on how to tag Japanese names or one appends the series name to the end while others don't. One site may have completely different unnamespaced tags for the same image, or the booru tagger may have made a typo during tagging... I feel like it's easier to just tag from scratch using the tags I know that I use instead of hoping that booru taggers can agree on a single tag for 'girl but with a pp', stop making typos, etc. Maybe I could use a ridiculous amount of tag siblings to fix those issues but I'd constantly be adding new siblings as I noticed more errors. >>20955 Booru shikkaku, by Anonu Devai

>>21097 It's a two part program, there's the GUI & the CLI, they're separate repositories. https://gitgud.io/thatfuckingbird/hydownloader is the main program https://gitgud.io/thatfuckingbird/hydownloader-systray is the gui

>>21094 edit the gallery page parser to remove the new "/en" part

>>21082 Hello Hydev, 14091 here. Thank you a lot for spending your time answering my questions. 1+2) Thank you for considering those features! 3) I have a lot of stuff already in Windows Explorer folders that i would like to import. In the beginning i wouldn't use hydrus downloaders alot. So that's why i kinda have to sync to the PTR to get all the mappings for files i already have. But thank you for reminding me of that importer option, i might use it in future too. Could you please answer this part here regarding this question: "I guess if this works like i just described, doing the same in the future again and again would mean only the mappings that are missing in 'my tags' would get updated and it would not start a huge job and migrate everything for every single file again, right?" So would it mean that the system is smart enough that in future migrations the job would be much faster because it only migrates the mappings for newly imported files and only few missing mappings for old files that got updated? This questions also applies to number 4) kinda, so would it be possible to just create three protable archives from Client1 for mappings, siblings and parents (and leave them there without deleting it, lets say i got enough SSD space), then import them in Client2 and in weeks/months i "update" the .db files (that i havent deleted, because i want to update them) on Client1 again, which would be much faster? Or wouldn't that work for the HTA .db files and they are overwritten/created from new anyway each time. If they are not updatable, maybe you could make them? Or course only if its not too much work. Im completey fine creating them from new, really. No matter what your answer is, i would probably try it out. Shouldn't be that frustrating for me personally but we will see. That it is possible at all is already pretty cool :) Thank you alot again, you are a legend! Happy new year!

>>21101 Hydev, me again. Forget the last part, i just read that you wrote "I don't think my HTA tech will update an existing file, but maybe it does. I wrote it years ago. Try it out!" I now understand that with "existing file" you meant the .db files, got it!

>>21098 >I don't use booru tags outside of basic stuff like copyright and characters because of how unreliable and varied booru tags are. Even characters are commonly fucked because different sites disagree on how to tag Japanese names or one appends the series name to the end while others don't. One site may have completely different unnamespaced tags for the same image, or the booru tagger may have made a typo during tagging... >I feel like it's easier to just tag from scratch using the tags I know that I use instead of hoping that booru taggers can agree on a single tag for 'girl but with a pp', stop making typos, etc. Maybe I could use a ridiculous amount of tag siblings to fix those issues but I'd constantly be adding new siblings as I noticed more errors. i see what you're saying, but keep in mind there's no reason you couldn't download the tags anyway. you can download them to a separate tag repository just as a holding area and make them not affect anything. set up your search and autocomplete to ignore that tag repository. that way it's like they don't exist in normal browsing. if you do this, you always have the option to pick and choose good tags to migrate from the booru tags to your tags at a later time. just because some of the tags are bad doesn't mean you should throw it all away. not downloading the tags seems like a waste to me. you're leaving data on the floor. why not hoard it all, even if you never use it?

>>21100 I mean, if these links do not expire, what is the issue that prevents using them for subscriptions?

>>21099 Yes I know, thatt is why I said I installed it then got the GUI and its al green because its supposed to be working, without the main thing installed and running it will not even launch

>>21103 Not him, but I do something similar with ai tags, where I put them to a separate service and turn off their display, but still keep them searchable, which helps with manual tagging, I just have to make sure to select a different service during search. My inly problem with this is that you can't select multiple services like with domains. It's either one or all.

>>21105 >>21097 Oh, I misunderstood, I think subscriptions only fire off after the set time has passed, there's no initial run to grab the files. If you run a regular download it should start working immediately. You can also run a check of what files are in Hydrus already with the hydownloader-anchor-exporter so that it won't download already downloaded files.

>>21095 Yes, the tag is created using the "modelname" line in the info.json for the tagger. changing this line would change the tag given to processed files. Alternatively, you could just sibling the tag back to the original (or vice versa assuming you only use one wd tagging model.)

>>21108 Good idea on the siblings. Thanks.

>>21107 Do I need to do any extra configurations outside putting the daemon key? This shit is just not doing anything for hours

>>21110 Also, I could use the anchor thing to import 400k hashes from hydrus but I can't somehow export the subscriptions from hydrus?

>>21079 >>21080 I don't know what to tell you other than that I'm not using the api, just scraping, and my subscriptions do work, and it's not redownloading everything every time. It works for me.

>>21112 > I'm not using the api, What does this mean

>>21110 apparently its lacking user/pass since its do loading nothing you could not see without acc but when I edit the config the stupid thing says syntax is wrong How is this supposed to look?

>>21113 The Hydrus API. Application Programmer Interface. A way to control Hydrus through a program, using functions supplied by Hydrus. >>21112 Out of curiosity, what downloader are you using? The Beta Sankaku one, or an old Chan. Sankaku one? I'm guessing you're connecting to some old backdoor that they haven't closed yet, that still uses the old chan rules, which had no keys or timeouts yet. Because yeah, we used to be able to download from them without any problem. Before they changed everything.

>>21114 I think you need to import your cookies from Sankaku. You can use Hydrus Companion or if you use Firefox you can use the extension "cookies.txt" no clue about Chrome, the cookie file should say: # Netscape HTTP Cookie File at the top. >>21111 I don't think so, and I don't think subscriptions are exposed through the Client API either. Maybe a possible feature request, Hydev?

>>20720 >Ultimately, I think siblings and parents need a complete visual overhaul. I wish we had nice auto-generated tree graphs. This script takes exported parents from clipboard (there is code for reading from stdin, but that's less safe), runs dot to generate a graph, and then mimeopen to display it. Requires the pyperclip Python module. #!/usr/bin/python3 import os import subprocess import sys import tempfile import pyperclip # get the exported siblings or parents (a list of lines representing a table) #exported = sys.stdin.readlines() # From stdin. Be careful not to paste dangerous commands into the terminal! exported = pyperclip.paste().splitlines() # From clipboard. # node names are enclosed in double quotes, escape them (not tested) exported = [s.replace('"', '\\"') for s in exported] table = zip(exported[::2], exported[1::2]) # make an actual table result = [] result.append("digraph {") # a directed graph for i1, i2 in table: result.append('"{}" -> "{}"'.format(i1.strip(), i2.strip())) result.append("}") dotinput = bytes('\n'.join(result), encoding='utf8') # sent to dot's stdin, it returns PNG data dotoutput = subprocess.run(['dot', '-Tpng'], input=dotinput, capture_output=True) with tempfile.NamedTemporaryFile(suffix='.png', delete=False) as png: png.write(dotoutput.stdout) pngname = png.name subprocess.run(['mimeopen', pngname]) # Open the viewer. input() # The next line removes the file, so the viewer may lose it. So we wait for user's Enter. os.remove(pngname)

I had a good week back after the holiday. I fixed some bugs and improved some system:hash predicate parsing. The release should be as normal tomorrow. I will try and catch up with the messages here early and make a new thread for it too.

>>21113 Also, he could mean Pixiv, Sankaku, etc. APIs, though I don't use anything like that. Any downloaders I make use Hydrus to parse the web page code, looking for tags, URL's, etc. That is all done within Hydrus.

>>21103 I suppose you're right. Only issue I can see is services getting cluttered which I find annoying. I'll give that a try, I can always delete the tag services later if I want.

>>21119 Dunno Anything I try, sankaku subscriptions just start again and redownload everything every single check Will this be fixed or plain give up on the site?

>>21085 Thanks. The mpv window is embedded into hydrus with duct tape and a prayer, so window manager bells and whistles can sometimes mess with it. You might like to play with the advanced window settings under options->gui->frame locations, the 'media_viewer' set. Maybe setting it to start as maximised/fullscreen--or setting it to not, if it is on--helps the initialisation and the window manager hijack here. Otherwise, let me know how you get on. If it comes back, is there any pattern to why it does? etc... >>21086 I think we spoke elsewhere about this, but in case that wasn't you, this is the latest quicksync afaik: https://breadthread.gay/ . It should have the update files in its client_file directory, which you can import under services->import repository update files, but you'll still have to talk to the PTR for a short time, on VPN or otherwise, to initialise your metadata store in the review services panel. >>21096 I'm not involved in running the PTR day to day anymore, but I haven't heard anything about this. I know plenty of Russian users have to use VPN to access all sorts of spicy locations like boorus, maybe it is the same, some odd ISP-level block to that data center? In any case, I generally recommend everyone use VPN these days, no matter what imageboard-tier stuff you are doing. >>21098 >It would be cool if we could do booru wiki style entries where you could link to other tags or even URLs, but to me the most important part would be ease of looking up a basic definition for that tag. Maybe type it in to the add tag box and right click a candidate tag and hit 'see definition' or something. Thanks--I think this is what I envision for a first version as well. Well done for finding your backup! You've already learned the lesson, but I'll still link you to the secret help page: https://hydrusnetwork.github.io/hydrus/after_disaster.html >>21101 >>21102 >I guess if this works like i just described, doing the same in the future again and again would mean only the mappings that are missing in 'my tags' would get updated and it would not start a huge job and migrate everything for every single file again, right? Yeah in general, I can and do skip this work when possible, and it is usually possible. My general default when there are 'overwrite' conflicts is just to merge, so if you throw 100 tags at a file, and it already has 80 of them, it'll simply add the new 20, no worries, and very fast. In general, 'I don't need to add this because it already exists' rows tend to work about 100-1000x faster than a new data row. So instead of taking 100 time units to do this work, it'll take like 20 + 0.08. My general 'it might take ages if you regularly repeat this' worries are related to it taking a bit of time to set up this job just when you have to click all the UI, and then also when you sync from a gigantic datastore like the PTR to a local store, there is a bunch of unavoidable overhead just from the files you don't have. The PTR has a couple billion mappings, of which maybe a few million might apply to your local collection, so that '100-100x faster' bulk suddenly gets a lot larger. Even if I make the 99.9% of negative matches work very fast, it is still going to probably be ten or twenty minutes of waiting around to go through them, or maybe even hours, and if all that work only adds 100 tags in some fresh sync, I generally think it isn't work it. Logically, though, there's no problem in running updates like this. >>21117 Thank you very much--I will play with this!

How about integrating gallery-dl to the downloader, this could help solve issues with ever changing hostile sites like Twitter.

>>21123 Apparently hydownloader does this but that shit barely works, it would be a godsend

New thread here: >>>/t/14270 I'll post v557 to it later today, and this thread should be migrated to >>>/hydrus/ soon. Thanks everyone!

>>21121 Nothing to fix in Hydrus. It's all on them. The Dev would have to do something internally with new ways of approaching the problem, and then they would just change again. I've had problems with subscriptions for Pixiv as well. Subscriptions work fine if it's just catching up with a few days. But beyond that, things seem to go weird. If you're going back further than that, you need to do a full gallery download, and just watch to see when it catches up (i.e. a lot of already in db status, showing you've caught up to where you stopped downloading the last time)