>>10992 —just pasting it here in the new thread too— here, this is a known link that doesn't pull images up, I tried going incognito to find ones, but apparently it finds the images that way too. for the artist, so I stuck with a known one that wouldn't work. https://mega.nz/#!KooGTIBC!aRlDdrI1XRA-iMDmkzye7J6Q-C3bTWnb3_74-8-0eDI Honestly, I think full stopping the download, program wide, till a log in finishes would be a good work around for this. or with the downloaders that these download from, a double tap when logging in, the first page or two of searches could get redone twice, because by then everything should be logged in, and if something didn't catch it would get them in the first two pages. ————— that said will install the new version right after some downloads finish.

>>11002 Got a minor graphical glitch of some sort >>11003

>>11002 Goodbye Youtube-dl, my friend. Hope I can see you when API comes

Deleted my download rules but my Tumblr subscriptions are still stuck on waiting for bandwith for 9 hours…have tried pause/playing them and a bunch of other stuff. Any way to make it just download before I run out of time?

lol, I could likely remove a good 500gb of this if I had a way to get a user note to show up in the import status section. welp, here's hoping next round that option is closer to the top.

I think I >>10818 screwed up my post back thread version 332. I just realized I deleted my reply to you by mistake trying to correct it so I'm not sure you was able to see it or not. From your reply >>10856 I did the other stuff and none of it worked. I made the html like you said and looked through it in notepad++ I didn't see anything about "not logged in". I made a pastebin https://pastebin.com/TtNtZnXi I'm not really worried about private info since you don't need an email to make an account. You just create a name and password and the site lets you can see everything.

>>11013 Jesus christ anon, what the fuck?

>>11015 Downloaded a bunch of artists I like, and have been watching nearly every thread on 4chan boards I like that look interesting since thread watchers got single tabbed. I want to go through this, I want to do the dup search which currently only sees little over half my database, and has a further probably 180k dups, and potentially even more once the two are combined, but for this I need the ability to see why something got deleted otherwise my autism will see a good file, wonder why the fuck it got deleted and assume is an accident, and any form of culling becomes pointless, and with the current dup system, once a file goes though, it doesn't go back though, which would make it even worse when I re import something that got deleted due to dup. right now i'm only sending things to trash, and then once there i'm going to only delete things that are irredeemable as an image and would never be mistaken with something I want but beyond this, its all I can do till I can see why something got deleted here. until then, I am kind of in a position of not being able to do anything. having quick shortcuts would be nice, but really I just need the skeletal work in the program, I suggested make user notes show up there, but nothing came of that so far i believe. the only necessary thing is once a more permanent solution is found, all the prior work isn't for nothing.

>>11005 Hey, thank you for this video. Please check what you have under network->downloaders->manage logins. I do not think you are logging in with your account, but instead the click-through login. Hit 'change login script' on www.hentai-foundry.com on that dialog and put in your credentials. I actually have some nice new help at the bottom of here on this: https://hydrusnetwork.github.io/hydrus/help/getting_started_downloading.html Also, I strongly recommend you clear out your completed download queues. Your downloader there has 400k URLs in it, and the client tends to get slow at about 40k total thumbnails/URLs loaded. That huge list is also going to be a big cause of program lag and other jank. There's the smallest chance it explains the login problem, maybe as some kind of traffic jam, but in either case, please clear out what is finished there–it'll fix a whole bunch.

>>11007 Thanks. This happens sometimes to the popup area, I think mostly due to my jank-ass way of making it appear. It is generally harmless. I'm going to see if python 3 improves things at all, and if not I'll revisit it. I'm also hesitantly entertaining a possible move to Qt for Christmas next year.

>>11008 Yeah, I discovered this week that it can do some video tweets. I really like it and use it every day for all my regular youtube subs. I'd love to one day have it as something the hydrus downloader can throw URLs at and get mp4s back, but the API will also be able to hook the two together. The old youtube solution, which used pafy, could only handle direct video URLs and not streaming DASH stuff.

>>11011 No force mode atm. Check what is 'blocked' under network->data->review bandwidth. Could be some tumblr CDN domain.

>>11011 Yeah, in that html is the login form, so it looks like your hydrus is not logged in to r34h. If you are up to it, I recommend going to network->downloader definitions->manage login scripts and trying to login manually with the shiemmie login with the test panel. Pic related. I just ran it with some test credentials and it seemed to go ok. Note you can double-click the log entries on the right and copy the exact data that comes back and review the cookies given, so please run the test and see what happens. Even though it thinks it is all OK, maybe the shm_user cookie it gives you is bad? Mine just now had a timeout of exactly 1 year–do you get that? Anyway, take screenshots of the panels and pastebin the page response for that last 'post login form' step, and we'll see if it includes any 'login failed m8' text. Note another option here is to copy the cookies from your browser straight to hydrus under netowrk->data->review session cookies. Since it lasts a year, this might be a decent fuck-it fix for now.

>>11003 requesting mr bones track how many files have no tags too.

>>11027 my understanding, at least the last few times I did performance testing, any lag/jank is related to open images, url and downloader wise, it comes down to number of active downloaders That said, I have been considering of a clean start once I go though the images and having everything there saved as a legacy go through… that will be gone though once I can do something about duplicates, which for me necessitates >>11021… but like I said, I have something around 700gb of space left and push come to shove I can make more space by removing the fuck huge files. as for the program not logging me in, it does, the 15 image is the click-through, the 68 image is the logged in, though it was still doing a guest log in, so not i'm logged in for 30 days at a time which is good, changing it from click through script to login though seems to have make the program wait longer though. >>11029 If you go the youtube-dl route, I have a request. Can you have an option in the downloader for it to NOT use the hydrus archive for saving the file and instead a different directory? While hydrus is great, its great for stand alone images, and would be fairly shit if I stored video in current hydrus. Its partly the reason I wouldn't use current hydrus for manga, while I would love the program to work with manga, I would have to rely on the program to work going forward, which I can't do. You as the dev put 334 updates and have been doing it near weekly, I trust that as long as you are alive you will keep supporting the program and make sure it works. but lets take my manga folder into consideration, I have around 2.5tb of manga and h-manga combined, possibly a bit more if I looked on more hdds, if I put these into hydrus, it would hash them, and make them non user sortable. if I just put images in, It would fuck 170 images+ from ever being read in sequence again, and lets put it this way, when it comes to me downloading manga, I would rather re download a manga then take the time to rename it to fit a convention, there is no way in hell I could do that for hashes. for single images? before I had hydrus, I had around 120-200gb of images, and another 500-1tb of gallery images that were never going to be sorted anyway, so I was no loss to me to hash them, if anything I lost may 30000 images out of my real/drawing/real porn/drawing porn sorting method but I believe enough in hydrus that it was no problem for me even if it failed horribly. For this reason, I wouldn't want to have youtube-dl save to the gallery, but on the same note, I would love a better ui for youtube dl then the crap that is currently out where the person behind it I think has fucked off and told anyone with issues that he doesn't see the issues as issues so pound sand. If you end up making a youtube-dl ui option, It would need a few things to it 1) video quality/format choice with a 'choose highest quality' option 2) audio quality/format choice with a 'choose highest quality' option 3) an audio/video only mode 4) at least for me to use it, an out of archive save location that saves based off of videos title In all honesty thinking of the manga portion again, If you ever do that, I highly suggest putting all the functionality of having a hydrus managed folder layout that is also user readable into the main client but having it walled off, that way a branched version takes no real work to manage as you are already doing everything with the main version anyway. Hell, You could have 3 folders inbox / archive / trash inbox - being where everything gets put in when it first comes to the program, archive - where everything goes once ok'ed, realistically it's a holdover from current hydrus, but I have had manga where the images are fucked and a v2 was needed trash - where everything goes if deleted. in the archive, things would be sorted from series - (manga name) or artist - (artist name) instead of the current has possibly splitting adult comics into its own little space with a series and artist separation there too. In the main non porn, series comes above artist, so if something is sorted into a series that's where the files go, with a possible symbolic link in artist, for the porn section, artist is where things go, with series being the symbolic link This way, while the program works, you would do all the reading and browsing in it, and when its fucked, people can VERY easily migrate to something else if they even choose to do that much. Hell the manga version could also do all video too, sorting it much the same way, with series being something more for tv/movies and artist being something more for youtube channels. I think I got very off track from what this was originally about.

Ok its been a month since I had my admittedly autistic sub made for derpy's 3 day best, and I got some data for it. first we go here showing the range surprisingly to me, even the top of the range finds new images, the bottom of the range has only ever went into 'yea, we have seen enough stopping search' twice which suggests that though all the fucking filtering, I may still loose files. will have to add a few more delineations along the way. with that said, it seems that the import has a cumulative number, as in watcher number 1 on that list will have a progress bar that is 1062, with new images added to the end, making the progress bar completely useless. would there be a way to add an option to just have it show the current images it found and is checking?

I have trouble opening the bugs thread, so I'll just dump my issue here. I still get pretty bad X Window-system crashes when running on my Antergos (Archlinux with some ease of use stuff) system (x86-64 with relatively new packages). 1. For example, when zooming on a large image, I get:
OpenCV(3.4.1) Error: Assertion failed (dsize.area() > 0 || (inv_scale_x > 0 && inv_scale_y > 0)) in resize, file /io/opencv/modules/imgproc/src/resize.cpp, line 4045

2018/12/27 00:04:46: Uncaught exception:
2018/12/27 00:04:46: error
OpenCV(3.4.1) /io/opencv/modules/imgproc/src/resize.cpp:4045: error: (-215) dsize.area() > 0 || (inv_scale_x > 0 && inv_scale_y > 0) in function resize
File "include/ClientGUICanvas.py", line 5620, in EventPaint
self._Redraw( dc )
File "include/ClientGUICanvas.py", line 5583, in _Redraw
wx_bitmap = self._image_renderer.GetWXBitmap( self._canvas_bmp.GetSize() )
File "include/ClientRendering.py", line 134, in GetWXBitmap
wx_numpy_image = ClientImageHandling.ResizeNumpyImage( self._media.GetMime(), self._numpy_image, target_resolution )
File "include/ClientImageHandling.py", line 353, in ResizeNumpyImage
return cv2.resize( numpy_image, ( target_x, target_y ), interpolation = interpolation )


The program 'client' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadAlloc (insufficient resources for operation)'.
(Details: serial 649355 error_code 11 request_code 53 minor_code 0)
(Note to programmers: normally, X errors are reported asynchronously;
that is, you will receive the error a while after causing it.
To debug your program, run it with the --sync command line
option to change this behavior. You can then get a meaningful
backtrace from your debugger if you break on the gdk_x_error() function.)
2. Haven't fully nailed down when, but I will sometimes get
client: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.0.
This mainly occurs when entering tags or when doing the archive/delete filter. May be a nasty multiprocessor bug, given it's an IO error. In both situations I have reasonable memory available (+4GB).

>>11119 Regarding no. 2: I've nailed it down to the "manage tags" window. The filter works nicely, but I cannot add tags while filtering, because that may crash hydrus.

Hey h dev… So, just got done making a split 4chan thing where I only have the active ones now and I have the inactive ones saved to be dealt with later. I went from a working 12-14gb down to 1.4gb of use… I still have around 20-30k images open that I have been getting around to… but holy fucking shit, what the hell was eating the ram? I know that last time I did performance testing, 40~k images what the hard limit on where the program will start stuttering, but I can't for the life of me think of what the fuck was eating 13~ gigs just from having threads in the watchers? at least the way I see it, there would be data on what image it is, where said image is, its tags and shit, all that fun stuff, and I cant come up with a reason that the un highlighted threads have any reason to eat up more then 100mb, much less 13 gb… able to explain this at all?

>>11128 Not dev, but were you archiving from /gif/? For reasons unknown to me hydrus runs ffmpeg every time it downloads a file. Checking for compatibility, reformatting for consistency, maybe? Could never figure it out, but every /gif/ thread I have a downloader entry or a tab for always seems to scale up mem use far more than the others, and I notice those wild ffmpegs. Maybe dev can explain better if this is the case.

>>11141 I think in total around 1.2 million links, with well over 75% of the files being things I already had, gif was one board, but I watched every board I removed dead watchers for just lighten the load a bit more, more so then getting around 40k images in active use removed. the fact it persisted through restarts was also an issue that makes what you described questionable if that was it. I mean I would understand it if every now and then a restart would reset everything, but at least as far as i'm aware everything should be text, and even assuming there is a good paragraph of text for each file there is no reason it should have eaten that much ram.

>>11052 Yeah, there's a bunch of these bad progress gauges, including repo processing, where it made sense in the early days or during devving and I am doing 17/25, but in reality a lot of this stuff ends up being 4000/4003. I'll make a cleanup job to go through this and make it more helpful.

>>11119 >>11120 Thank you, I will check manage tags during filtering! The zoom problem is probably unavoidable for now. The rendering pipeline is still shit for this as it ends up making a whole giant bmp rather than tiles, so zooming to virtual 20,000x20,000 screen real estate actually tries to allocate that much ram, breaking a whole bunch of stuff. It's another thing I have to fix. "Don't zoom big stuff 800%" is the unfortunate solution for now.

>>11128 >>11141 >>11145 I am not sure what the superbloat was there. There's a few indices being built with every import list, so that adds a little bit, but it usually only gets killer when there is one big list, like 1 x 600k, rather than 10 x 60k. I think I've said before I hope to add a memory profiler in py3. It may be I am accidentally holding on to additional copies of some information somehow. It is possible some of your saved memory here is helped by my new 'media results' cache, but I'm a bit in the dark on the details until I can profile this properly. I use ffmpeg to figure out video mime and pull a thumbnail, so if the client can't guess a new file is a jpg or whatever straight off the first few header bytes, it gets sent to a new ffmpeg to see what it thinks of it. If you are importing a bunch of webms in a row, you'll be burning a decent whack of CPU. I understand there were once some ok python libraries that wrapped the ffmpeg dll and let you load it properly, but I don't know of a good one now, so I have to rely on some mickey-mouse subprocess piping bullshit to get all this to work (including video rendering, wew!). It shouldn't increase hydrus memory as it'll be a child process, but maybe your task manager counts up total memory usage including child processes or whatever. Please let me know how py3 goes for you here–it hopefully will be better.

>>11158 So I have just been doing some massive housecleaning imports over the past few hours, I added about 8000 files to active use, but have processed probably close to 100gb of files, almost all but the 8000 images going straight to the trash because they were already in the db. I think I hit a point where hydrus just acts fucking retarded. so, I went from clearing out all of my old 4chan dead threads, this allowed the program to go down to 1.3gb of ram use, and trhough adding new pages, and a few imports up to now, it had kept it around 2.3 to 3gb, a reasonable uptick for not restarting the program at all, but then I did the importa, and everything is going good, it went from 2.3 to the 3.3 gb range and it processing files and all that shit, ok I get it, its going up but now… its pretty much stopped importing things, I am on ninjakitty, a 7500 image import, and holy shit is hydrus thorwing a fucking fit. ur went from 3.3 to 12gb then it capped out my ram for a bit, thats not in the image, and then debloated, only to then rebloat to 15gb of ram and debloat It is now pinging my 1700 to 85-95% use and is currently 1.8gb of ram once it's done with this import im restarting it, but jesus christ did it bug the fuck out for no discernible reason. can't wait for the new version, here's hoping every issue i ever had gets solved.

>>11186 lol for all the bugfuckkery it just did, it added a grand total of 56 images of 7000 some potential ones to the archive.

>>11187 down to 1.8gb

ok got a bit of an issue not able to connect to hentai foundry and newgrounds im not sure has ever worked. 4chan downloading works, so its not something inherently broken with network, Could not connect!… (Copy note to see full error) Traceback (most recent call last): File "include\ClientImportGallerySeeds.py", line 249, in WorkOnURL network_job.WaitUntilDone() File "include\ClientNetworkingJobs.py", line 1091, in WaitUntilDone raise self._error_exception ConnectionException: Could not connect!

Ok, I mentioned this issue a while back but it got dismissed, however it cropped up again when importing files the beginning will start to bloat, where in a fresh directory it will be 0, after a while it will start to add numbers to the front. it's possible that files that cannot be imported add to the beginning, but the issue is once removed, they still persist right now from my image importing I had a clean folder but it has gotten around 140 images/files I can track that that couldn't be imported. but it starts at 185. is there a way to reset this/allow the importer to forget previously impossible files to import?

>>11201 HF seem to have let their ssl cert expire (but only as of Jan 6th, I think?), I don't know what is going on over there atm. Maybe they blocked hydrus based on User-Agent, but maybe their server is undergoing some other fuckery. I will keep an eye on it and see if their get their https back in order. I recommend you pause any HF subs/downloads for now.

>>11203 Hey, I am sorry, I do not totally understand what you wrote here. Can you rephrase it or add a screenshot? I do remember a user reporting an issue with 'import folders' having bad tracking for the files it did not import (for whatever reason)–was this you? If you check the button next to 'review currently cached import paths' on the edit import folder panel, what do the paths there suggest about your problem?

>>11213 oh simple. this is from a different importer where my hand saved images goes, so its showing 0 extra files. however before I moved it to its new location, despite the folder being empty it would display 347/(however many new images where there here) I think that it sees failed images and adds them to the beginning but regardless if you clean the images out, it sticks around, since I have been doing some cleanup on the g drive were my torrent folder is I have been going though artist archives and importing them, now there was one that had weird as fuck formats that were not importing and it skipped them, it added quite a large number to the beginning of the count, and subsequent imports have been adding 1-2 to the number due to txt documents, despite the images being removed from the folder. at the time I didn't realize this was likely the problem and I believe you though that it was a folder that just had all those images in it and they were never moved.

>>11226 here found a small artist I didn't import yet this screenshot was taken before any imports happened.

>>11227 >>11226 Thanks. I will make a job to check this code and have it occasionally clear out paths that no longer exist. I'll also be looking at changing some of these popup progress gauges to show progress more for the 'current job', like 1/17, rather than always showing entire-life progress like 4001/4017, which is always just a green bar with one pixel of grey, no matter how big the most recent sub sync or whatever was.

>>11231 lol asked about that somewhere too because that was a bit annoying seeing 'oh there are 1000 images here, the fuck happened' only to see the next stage of the sub have 975/983 and realized it remembers everything. not worth resetting the checkers for as to me it was a minor annoyance. I should also say, the weirdness where on saving the session where it would bloat from 12gb to 18+gb has been solved with the removing the dead watchers from active view, however with a little over 100k images in active view mode, I am unable to set it to every 5 minutes as it does start to bloat and lag a bit, no where near the extent it did before, but a bit.