When downloading a deviantart gallery, hydrus throws out an exception if it fails too many files. This is a pain in the ass if the files are legit, but cannot be scraped, such as http://white-angel-ariah.deviantart.com/art/Princess-of-the-Silk-Road-144206154 http://white-angel-ariah.deviantart.com/art/Lullaby-for-Lost-Souls-145623846 http://white-angel-ariah.deviantart.com/art/Song-of-Temadon-145912023 These count as failed images for hydrus, but they're just text. It makes no sense for it to fail them and count them towards the max fails. It should just skip them.

>>5115 Thanks–I'll fix this to be a delete_only action. >>5119 Thank you for this report. I will catch these errors better and return 'uninteresting mime' instead of the 'failed' or whatever they currently give.

>>5127 Also apparently "wide" and "tall" are switched in the sorting order. Sorting by "Least wide" for example gives me wrong results. Pic related, and pics attached.

Not sure if this is something that i should email the dev about, but.
DataMissing
Tag error in database
Traceback (most recent call last):
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.HydrusDB", line 467, in _ProcessJob
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDB", line 10139, in _Write
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDB", line 3437, in _ExportToTagArchive
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDB", line 5794, in _GetTags
DataMissing: Tag error in database
I've been trying to save all my tags in .db form every so often, because the last time hydrus borked up for me all the files were still there, and having a 40MB backup is easier then having a 450GB one. Recently updated to the recentest hydrus, 244a, and this happened alongside the backup failing. Possibly might be related to an earlier issue where hydrus panicked over something called a Zero-length Tag which i left well alone and didn't try to go back to fix.

Hi. Not shure if it's bug or not, but if you select same directory for file storage in options then Hydrus starts deleting files. For example, by default Hydrus has path like this "C:\HYDRUS~1\db\client_files". But if you add same directory like this "C:\Hydrus Network\db\client_files" and press "rebalance file storage" then Hydrus start deleting files.

pixiv not working on Linux, SSL error http://pastebin.com/v5CLh4aL workaround: set PYTHONHTTPSVERIFY=0 env. var.

>>5129 Thank you for this report. I will make sure to fix this as soon as I am back to normal work. >>5131 Thank you for this report. I will make sure to cope with this error better once I am back to normal work. >>5134 Someone else mentioned this to me just this past week. This is a serious bug that I did not intend. I will make sure to fix it as soon as I can. Please avoid adding synonymous names to that dialog until this is fixed. >>5155 Thank you for this report. I will look into this when I am back to normal work. Do you have any other SSL issues with your computer? Would you prefer if I added a 'don't verify ssl certificates ever' option?

>>5161 Hey dev, there's a race condition in case you try to manually import files from a folder that you set as monitoring. If the program decides to import from the folder while you're already importing those files manually, you get
IOError
[Errno 2] No such file or directory: u'/home/ayylmao/hydrus/general/optimized_2017.02.18.36.071674895.png'
Traceback (most recent call last):
File "/home/ayylmao/hydrus/include/HydrusPaths.py", line 447, in MirrorFile
shutil.copy2( source, dest )
File "/usr/lib/python2.7/shutil.py", line 130, in copy2
copyfile(src, dst)
File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: u'/home/ayylmao/hydrus/import/optimized_2017.02.18.36.071674895.png'
It's probably wise to make the program suspend folder monitoring for as long as the manual import screen is active, kinda like how subscriptions don't work if the subscription folder is open.

>>5174 >/home/ayylmao/hydrus/general >/home/ayylmao/hydrus/import These are the same directory btw, I just derped while trying to hide my username.

This is a weird little bug. I'm on linux btw. >pick a page full of files >right click -> select -> pick a category that does not have the first file of the page included in it >press enter >the first file of the page is selected instead of the first file of the selection

>>5178 >>5178 >>5183 Thank you for these reports. A lot of this stuff is asynchronous, so I'll clean up recovery from missing import files in general, no matter the context. I'll look into the selection issue as well. I expect I missed setting up the 'hidden' focus properly on that kind of selection event.

Getting a weird issue with the windows manager i3 where the dropdown box that normally occurs when you make a new Files tab doesn't show up, as well as suggestions based my current typed answer don't show up below. I can still type tags in and get results, though the only system tag that works is "system:" which results in "system:everything", which is hardly useful

I accidentally added an empty namespaced tag to a file in the tag manager window, it was "page:". Now I get this error whenever I try to open that file's tag manager window again, and the window doesn't come up.
DBException
SizeException: Received a zero-length tag!
File "include\ClientGUIMenus.py", line 106, in <lambda>
l_callable = lambda event: callable( *args, **kwargs )
File "include\ClientGUIMedia.py", line 865, in _ManageTags
panel = ClientGUIScrolledPanelsManagement.ManageTagsPanel( dlg, self._file_service_key, self._selected_media )
File "include\ClientGUIScrolledPanelsManagement.py", line 2827, in __init__
page = self._Panel( self._tag_repositories, self._file_service_key, service.GetServiceKey(), self._current_media, self._immediate_commit, canvas_key = self._canvas_key )
File "include\ClientGUIScrolledPanelsManagement.py", line 3069, in __init__
self._suggested_tags = ClientGUITagSuggestions.SuggestedTagsPanel( self, self._tag_service_key, self._media, self.AddTags, canvas_key = self._canvas_key )
File "include\ClientGUITagSuggestions.py", line 434, in __init__
related_tags = RelatedTagsPanel( panel_parent, service_key, media, activate_callable, canvas_key = self._canvas_key )
File "include\ClientGUITagSuggestions.py", line 207, in __init__
self._QuickSuggestedRelatedTags()
File "include\ClientGUITagSuggestions.py", line 238, in _QuickSuggestedRelatedTags
self._FetchRelatedTags( max_time_to_take )
File "include\ClientGUITagSuggestions.py", line 227, in _FetchRelatedTags
predicates = HydrusGlobals.client_controller.Read( 'related_tags', self._service_key, hash, search_tags, max_results, max_time_to_take )
File "include\HydrusController.py", line 260, in Read
def Read( self, action, *args, **kwargs ): return self._Read( action, *args, **kwargs )
File "include\HydrusController.py", line 93, in _Read
result = self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
File "include\HydrusDB.py", line 696, in Read
return job.GetResult()
File "include\HydrusData.py", line 1998, in GetResult
raise e

Database Traceback (most recent call last):
File "include\HydrusDB.py", line 466, in _ProcessJob
if job_type in ( 'read', 'read_write' ): result = self._Read( action, *args, **kwargs )
File "include\ClientDB.py", line 7453, in _Read
elif action == 'related_tags': result = self._GetRelatedTags( *args, **kwargs )
File "include\ClientDB.py", line 5376, in _GetRelatedTags
tag_ids = [ self._GetTagId( tag ) for tag in search_tags ]
File "include\ClientDB.py", line 5833, in _GetTagId
HydrusTags.CheckTagNotEmpty( tag )
File "include\HydrusTags.py", line 146, in CheckTagNotEmpty
raise HydrusExceptions.SizeException( 'Received a zero-length tag!' )
SizeException: Received a zero-length tag!


2017/03/02 09:35:55: Traceback (most recent call last): File "include\ClientController.py", line 1110, in THREADBootEverything self.InitModel() File "include\ClientController.py", line 525, in InitModel HydrusController.HydrusController.InitModel( self ) File "include\HydrusController.py", line 203, in InitModel self._db = self._InitDB() File "include\ClientController.py", line 64, in _InitDB return ClientDB.DB( self, self._db_dir, 'client', no_wal = self._no_wal ) File "include\ClientDB.py", line 161, in init HydrusDB.HydrusDB.init( self, controller, db_dir, db_name, no_wal = no_wal ) File "include\HydrusDB.py", line 244, in init raise e Exception: Updating the client db to version 244 caused this error: Traceback (most recent call last): File "include\HydrusDB.py", line 223, in init self._UpdateDB( version ) File "include\ClientDB.py", line 8965, in _UpdateDB self._c.executemany( 'INSERT INTO tag_sibling_petitions ( service_id, bad_tag_id, good_tag_id, status, reason_id ) VALUES ( ?, ?, ?, ?, ? );', new_inserts ) IntegrityError: UNIQUE constraint failed: tag_sibling_petitions.service_id, tag_sibling_petitions.bad_tag_id, tag_sibling_petitions.status

Following this, the program crash. I can restore to 243 and work. For context, I have at least 143k uncommitted tag, due to some unvetted filename. I'm not smart enough to understand what these line of code mean either, sorry.

Adding a space in between tags and namespaces appears to be borked. 244a used to be able to handle this properly and 245 doesn't.

>>5254 Read the changelog for 245. >- leading and trailing spaces are now removed from both the namespace and subtag components of namespaced tags, meaning 'title: blah' will be collapsed to 'title:blah'. tag repositories will clean their existing tags on update

>>5254 Also >that title bar font Jesus fucking christ dude.

>>5257 Hey, i like Caslon. It's the Monkey Island font and it brings back happy childhood memories. Also, this update is breaking my workflow. I import a large amount of tags from the public repository wholesale, and have difficulty telling which tags are mine and which tags are copied from other sources. At least add an option to disable or enable leading and trailing spaces as needed somewhere discrete.

>>5258 Quick update. If you had a tag with a leading space in hydrus 244a (And presumably earlier, but i haven't checked), updated to hydrus 245, and tried to remove or delete your old tag from a file, Hydrus will remove it from your tag editing window but not from the database. A quick refresh of the page confirms that all of the tags that have leading spaces aren't removed at all. Since over half of my tags have leading spaces that i can't touch, get rid of, or change in any way, Hydrus is unusable for me right now.

>>5226 Under options->gui, what's your always embed autocomplete dropdown results window set to? It should default to True on Linux, but perhaps it has flipped to False or Linux detection failed somehow. The default behaviour for the dropdown, on Windows, is to be a floating window that only appears when the text box is selected. It looks like that's what you have there–but displaying the floating frame is obviously buggy on Linux. >>5232 I apologise for this. I have made such tags invalid. I updated the server db's tags, fixing them where possible or wrapping them in 'hydrus invalid tag:"'', but did not do the client. I will be updating the clientside bad tags for v246, so I hope this bad tag will be similarly changed on Wednesday. Please let me know if it still breaks at that point. Did you add this tag in v245 or a previous version? >>5249 >>5250 Thank you for this report. I apologise for the inconvenience. I am not sure why this is happening–I presume you have pended or petitioned the same tag sibling more than once or with different reasons and my code did not clean it up correctly inside the db. I think I have fixed it in the v246 release. Please let me know if you still get problems at that point. >>5254 >>5256 >>5258 >>5263 I apologise for this. It is the same problem as >>5232 . Now I have made such tags invalid to enter, you can't fix them manually. I will collapse all these tags clientside in v246. When I can do it simply, I will collapse a 'namespace: subtag' tag to 'namespace:subtag'. When I cannot do that for complexity reasons, it will be replaced with 'hydrus invalid tag:"namespace: subtag"', which is a valid tag in the new system. If I have time, I will also add a gui option to automatically render all namespaced tags with this leading space (even though internally, the subtag with have its leading spaces removed). If i don't, it'll be in for v247. As you are a user who wants this leading space, please keep me updated on how it works for you in future versions. If you end up with a million 'hydrus invalid tag's and it is way too much to fix manually, then let me know and I'll see if I can knock up a script to fix them or something.

>>5269 >Did you add this tag in v245 or a previous version? It was v244. I tried it again on v245 and same thing happens. Now I've got two images that won't let me edit tags :P

>>5269 Thanks boss, you're the boss. I'll see how 246 fare then. I tried some stuff in the meanwhile. I created a back up, discarded all the commit and then successfully updated to 245 confirming the commits really where the problem. I restored, downgraded to 243 and now am in the process of checking the parents, sibling and petitioned tag. Perhaps it is a file whose tag I petitionned but ended up deleting? No dice thus far.

>>5269 >>5254 >>5256 >>5258 >>5263 Following up on all of these, just made the switch from 245 to 246, so this will be first impressions only. First off, i prepared for the switch. I've mentioned before my reasons for adding leading spaces to certain tags was because i like to both save files from 8chan with their original filenames and sync with the public tag repo, and i like having a easy, at a glance way of telling weather or not any tag was created by me or by some anon on a webm thread being clever. With that in mind, before i made the switch, i moved across every single file that has only tags with leading spaces into my archive folder, and anything that doesn't, i kept in my inbox. Took me a while to do, actually. With 245, it was hit or miss weather or not tags with a leading space would autocomplete or not, so i had do to a lot of clever tricks by using - searching to narrow down everything that works before moving it all across. With that done, and with all 80,000 untainted files moved across, i made the switch. I can say without fear of exageration that literally every single file in my database now has an "invalid tag:" subtag on it, because two of the ones that didn't carry across was "medium: image" and "medium: video", which between the two comprised every file on my database. But Autocomplete now works like it used to, I can save my tags to external DBs again, even submit them all back to the public tag repo when i thought i couldn't since v244, and my dog came back to life. Good job, Hydrus dev.

>>5289 >>5289 >was "medium: image" and "medium: video" So it was you who created these… Why, pray tell, when we can sort by mime already? Those could work but only as encompassing tag type if we had tag group. (as in: 'Franchise:tag' silently searchable for every 'anime:tag', 'visual novel:tag', 'video game:tag' or 'series:tag' we add, because they are part of the franchise tag group.) I'm may "read" judgemental but I am legitimately curious.

>>5292 It's part of my workflow. When i import files, i always import a filename, then manually go through the imports and manually edit out anything with a wierd name, or one of Codemonkey's hexadecimals that don't actually mean anything. Then, and only then, do i add medium tags. I don't really care about the tag itself, because like you said, we have mime searches. But it helps me keep track of imported files, gives me a second chance to look through all of them, remove stuff that i don't like, add some tags that could fit on an individual basis. The medium: image tag itself doesn't matter. It's just a sign that this image has value to me and shouldn't be deleted.

Oh, and i forgot another thing i do; Anything that's a gif has to go through a pretty stringent set of requirements to stay a gif. If it doesn't move, it would work better as a jpg. If it's huge, it works better as a webm without sound. Since i'm always adding gif tags to everything, it's easy to check offhandedly if they meet my requirements or not.

If you try to delete many files from the trash immediately, like hundreds, you get an error about too many somethings to unpack.

Reminder that if you have a weird tagging thing you do that's going to be a headache for people n the repo you can always just tag it in local tag db instead. In fact that's what you should probably always do, and then submit tags to repo only if you're sure they're really useful.

>>5320 Thank you for this report. This is fixed in tomorrow's release. Let me know if it still gives you any trouble. >>5274 I think this might have ulimately been a typo. I've fixed it properly for tomorrow's release, if you still have this problem. >>5289 I apologise for the lingering mess. I had hoped to clean and convert these tags more successfully. I'll be rolling out service reset in the next couple of weeks, and I suspect resetting the PTR may clear some of this stuff out as it actually no longer exists serverside. Local tags are still a pain, although they are easier to manually fix, either by mass replacing or using some local siblings. Again, if any of this stuff proves overwhelming to fix, let me know and I'll see if I can make a tool to do it.

Hydrus refuses to start on linux. Running under a terminal spits out this.
Traceback (most recent call last):
File "<string>", line 20, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientController", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientCaches", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDefaults", line 2, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientData", line 4, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientFiles", line 7, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.HydrusFileHandling", line 15, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.HydrusVideoHandling", line 7, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy", line 180, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.add_newdocs", line 13, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.lib", line 18, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.lib.polynomial", line 20, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.linalg", line 51, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/numpy.linalg.linalg", line 29, in <module>
File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 420, in load_module
ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.23' not found (required by /home/patxi/Downloads/hydrus network/libquadmath.so.0)


>>5336 Do you have glibc installed?

>>5336 Looks like your running hydrus_dev's binary release of hydrus for linux, you're better off installing a package for your distro or running from source especially if you're not using Ubuntu. Looks like what you have is a version mismatch of some shared object's dependencies which would be sort of expected if you aren't using the same distro+version combo of Linux as hydrus_dev. If you're using Arch there's a package on the AUR: https://aur.archlinux.org/packages/hydrus/ I really ought to share my packages for Debian and ebuilds for Gentoo at some point, just haven't gotten the time. Running from source is a bit more difficult especially if you're not familar with python but there is a manual page for it: https://hydrusnetwork.github.io/hydrus/help/running_from_source.html

>>5362 It turns out this guy got it right. Linux Mint 17.3 ships with GlibC 2.19, where Hydrus requires 2.23. And updating it is a massive pain, so i just made the switch to 18. Which is annoying, cause SystemD and muh /tech/ cred. Issue resolved, i guess.

>>5371 Serves you right for 1) using linux but not with a rolling release distro 2) not updating as soon as possible


ZeroDivisionError
float division by zero
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 862, in EventPaint
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 771, in _Redraw
File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 686, in _GetXFromFrameIndex
Hydrus is trying to divide by zero. Please stop doing that, Hydrus, lest you succeed.

Got this 2017/03/28 15:56:02: C:\HYDRUS~1\PIL\Image.py:2244: DecompressionBombWarning: Image size (161475000 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack. from trying to import the full size png of https://yande.re/post/show/340865 Would be nice if Hydrus could just throw and error and give up instead of taking up all ram and locking up the system.

>>5401 Thank you for this report. I have fixed this for tomorrow's release–if you can remember, did this animation/flash file have only 1 frame, by any chance?

This is maybe not technically a bug as much as a ux concern, but "delete on successful import" apparently means import everything and THEN delete the successes. Turned into a bit of a nasty surprise when I imported a 250GB folder from the same drive and had to rush to move other things off that drive temporarily to give free space for it to finish.

Dragging an image from the client to Discord does not seem to work.

>>5416 That's a discord bug, not hydrus'.

>>5405 Thank you very much for this report. This was not intended, and I apologise for the inconvenience. Import folders will now clean up their paths every 100 files. >>5416 >>5421 Yeah, we don't know why this happens. It affects some other programs trying to drag onto discord as well. My current suspicion is it is a security/permissions issue, where discord doesn't trust DnD events from some class of other applications. For now, please DnD the file onto an interstitial explorer window, or your desktop, and then drag it onto discord.

>>5453 The importer is very slow, and behaves strange when asking it to "import tags from nearby .txt file", and that file is actually an image file. Also related feature request: Either allow {hash}.txt as tag file for a {hash}.{extension} file, or add a reminder that "paths include the file extensions", to the explanation of the feature. Either of the above would have saved me hours today.

I don't know if it's really important but when I start hydrus it always says in terminal: (client:$PID): Gdk-CRITICAL **: IA__gdk_window_get_origin: assertion 'GDK_IS_WINDOW (window)' failed where $PID is the PID.

>>5497 Thank you for this report. I apologise for the inconvenience. I have improved the non-text-file detection and reporting, and I have added a new help button to better explain how the .txt stuff works. >>5519 I am not totally sure about this stuff. I do not print it, and it happens at a lower level than I generally have control over. I am guessing I am instantiating a window in a way that GDK ultimately doesn't like, but it could also be a problem with the larger gui library, wx. There are not-dissimilar reports on the OS X console (mostly complaining about fonts), while Windows is much quieter (but maybe they are just being swallowed). I'd like to fix this stuff, but I don't quite have the dev setup or experience with Linux to quickly and precisely track it all down. If another Anon knows how to track it down to a specific call, I'm very happy to rearrange whatever it is that I am doing wrong. I don't know if by critical they mean "this is a big deal causing a memory leak" or "something was -1 when I would have preferred 17".

>>5536 Thank you for your answer. I don't know if it is any help but a quick search gave this page: https://bugzilla.mozilla.org/show_bug.cgi?id=539138 I don't know anything about this stuff but it might help? Also the error doesn't show up in any log in /db.

The danbooru downloader doesn't work right for me, when I try to download an image that has a larger original image I want the larger original image but I get the smaller image. Is there any way to get around this?

>>5553 Yeah it's been like that for a while now. You'll have to wait for the downloading engine, which will come after the dev finishes the duplicate filter.

I was letting hydrus(253) update and it threw a traceback when an update failed to import:
DBException
DataMissing: Service file hash map error in database
Traceback (most recent call last):
File "/home/username/git/hydrus/include/ClientServices.py", line 1282, in SyncProcessUpdates
( did_some_work, did_everything ) = HydrusGlobals.client_controller.WriteSynchronous( 'process_repository', self._service_key, only_when_idle, stop_time )
File "/home/username/git/hydrus/include/HydrusController.py", line 386, in WriteSynchronous
return self._Write( action, HC.LOW_PRIORITY, True, *args, **kwargs )
File "/home/username/git/hydrus/include/HydrusController.py", line 117, in _Write
result = self._db.Write( action, priority, synchronous, *args, **kwargs )
File "/home/username/git/hydrus/include/HydrusDB.py", line 824, in Write
if synchronous: return job.GetResult()
File "/home/username/git/hydrus/include/HydrusData.py", line 1662, in GetResult
raise e
DBException: DataMissing: Service file hash map error in database
Database Traceback (most recent call last):
File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 9396, in _Write
elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 7211, in _ProcessRepositoryUpdates
did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
File "/home/username/git/hydrus/include/ClientDB.py", line 6222, in _ProcessRepositoryContentUpdate
hash_ids = self._CacheRepositoryNormaliseServiceHashIds( service_id, service_hash_ids )
File "/home/username/git/hydrus/include/ClientDB.py", line 677, in _CacheRepositoryNormaliseServiceHashIds
raise HydrusExceptions.DataMissing( 'Service file hash map error in database' )
DataMissing: Service file hash map error in database


Database Traceback (most recent call last):
File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 9396, in _Write
elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 7211, in _ProcessRepositoryUpdates
did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
File "/home/username/git/hydrus/include/ClientDB.py", line 6222, in _ProcessRepositoryContentUpdate
hash_ids = self._CacheRepositoryNormaliseServiceHashIds( service_id, service_hash_ids )
File "/home/username/git/hydrus/include/ClientDB.py", line 677, in _CacheRepositoryNormaliseServiceHashIds
raise HydrusExceptions.DataMissing( 'Service file hash map error in database' )
DataMissing: Service file hash map error in database

So the ptr sync hasn't been showing when it's running. So then I try to do something and it freezes up. The problem here is that it freezes up EVERYTHING, I can't even ctrl+alt+del. Log doesn't show anything unusual, just that it was processing updates at roughly the same time I was trying to do a search. It's happened at least 3 times now where it freezing completely locked down my computer until whatever was going on was finished.

>>5538 Thanks for this link. I will be looking at updating to the new version of wx in the next few weeks. It is a big update, so it is possible these errors will disappear. If not, that will suggest it is my code doing it after all and I will try to better chase it down. >>5553 >>5562 Yeah, please hang on for this! >>5597 I am sorry you have had this. Problems similar to this have happened for several users now. Has your client recently been roughly closed while processing? Maybe a power-cut, or you force killed it? You are on Linux, right? Which flavour? This error isn't supposed to be possible, but I think a hard process kill at the wrong time is introducing database invalidity. Please pause your repository synchronisation for now, but I will write a maintenance routine to properly catch and set up a fix for this for next week. Please re-enable it in v255 and let me know how it works for you. >>5609 I am sorry for this problem. Please pause your repository synchronisation for now. When is the processing running? Just popping up for normal idle time maintenance? What are your settings for that in options->maintenance and processing? What happens if you start processing manually from services->review services? Does the system slow down in the same way immediately, or can you cancel it from the popup ok? Are you on Windows? What resources does Task Manager (Ctrl+Shift+Esc) think hydrus is using during this manual processing?

>>5635 Just normal idle maint. On now general browsing within 2 minutes, no mouse movement within 10, and no CPU usage above 50%. Don't remember changing those. The system responds normally if I don't touch anything in the client, but if I try to do a search or something (while unaware that it's processing), everything freezes up save for my cursor. Ctrl+Alt+Del does nothing until the processing has finished for that moment. Process now button is grayed out right now. Actually thinking about it, it's happened almost any time the DB is locked. I was going to test it with the duplicate search, but it's already got all the information. And if that does cause it to lock up (iirc only the client locked up when processing duplicates and trying to click on anything) I don't really feel like losing my computer for over an hour while it runs.

>>5637 So memory usage isn't always going back down properly, I'd been doing stuff all day and ended up with only 1 tab, searching the inbox for untagged files, searching my files and all tags. Only 200 files and memory usage was nearing 3GB. Nothing else was happening at the time, and trying to delete things froze the client up for just under a minute each time. Restarted the client and did the same search, memory usage is fine now. I had been doing huge searches earlier, fixing PTR stuff though so almost no thumbnails were being loaded at all.

Recently gelbooru imports / subscriptions stopped working for me. Import error:
File not found.
… (Copy note to see full error)
Traceback (most recent call last):
File "include\ClientImporting.py", line 209, in _WorkOnFiles
downloaded_tags = gallery.GetFileAndTags( temp_path, url, report_hooks = [ self._file_download_hook ] )
File "include\ClientDownloading.py", line 1027, in GetFileAndTags
( file_url, tags ) = self._GetFileURLAndTags( url, report_hooks = report_hooks )
File "include\ClientDownloading.py", line 1013, in _GetFileURLAndTags
html = self._FetchData( url, report_hooks = report_hooks )
File "include\ClientDownloading.py", line 677, in _FetchData
return HydrusGlobals.client_controller.DoHTTP( HC.GET, url, request_headers = request_headers, report_hooks = report_hooks, temp_path = temp_path )
File "include\ClientController.py", line 386, in DoHTTP
def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
File "include\ClientNetworking.py", line 357, in Request
( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
File "include\ClientNetworking.py", line 323, in _DoRequest
return self._DoRequest( new_method, new_location, new_path, new_query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, num_redirects_permitted = num_redirects_permitted - 1 )
File "include\ClientNetworking.py", line 323, in _DoRequest
return self._DoRequest( new_method, new_location, new_path, new_query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, num_redirects_permitted = num_redirects_permitted - 1 )
File "include\ClientNetworking.py", line 283, in _DoRequest
( parsed_response, redirect_info, size_of_response, response_headers, cookies ) = connection.Request( method, path_and_query, request_headers, body, report_hooks = report_hooks, temp_path = temp_path )
File "include\ClientNetworking.py", line 925, in Request
return self._DealWithResponse( method, response, parsed_response, size_of_response )
File "include\ClientNetworking.py", line 494, in _DealWithResponse
elif response.status == 404: raise HydrusExceptions.NotFoundException( parsed_response )
NotFoundException: File not found.
Maybe they changed page layout, or something like that. I hoped the update would fix that (having updated for a month) but nope.

New user here so dunno if I'm hydrus'ing right yet but file import fails to recognise mime type on filenames with unicode letters in it.

>>5661 It probably has to do with how all images are blocked if you're running an ad blocker.

>>5664 >go to gelbooru >asks me to disable adblocker >do so >still don't see ads because of pi-hole Gelbooru will never, EVER get my money.

>>5661 Same, gelbooru's broken here too. Fucked up half my subscriptions.

gelbooru's also fucked for me something changed about gelbooru that broke it for hydrus, unquestionably. e621 works fine so it isn't boorus in general

>>5661 >>5671 >>5675 Might have something to do with how gelbooru now doesn't "direct" link to their images in gallery queries they now do this silly link to http://gelbooru.com/redirect.php?s=<base64 encoded actual url>. Also there's no validation on the url as to whether it points to gelbooru so things like redirecting to arbitrary sites works. Thanks for the link extender gelbooru? Honestly I'd recommend just not scraping from gelbooru (pretty much all of their content is mirrored from danbooru anyway) as they are just going to keep fucking doing this dumb bullshit because "muh ads".

>>5676 Probably should have confirmed the problem before posting but yup, it's what I thought, hydrus can't handle the odd redirect url.

>>5676 >Honestly I'd recommend just not scraping from gelbooru (pretty much all of their content is mirrored from danbooru anyway) Danbooru has loli behind a paywall tho. And Gelbooru is by far the most common result on iqdb.

>>5676 Problem is, most of us are fucked now because gelbooru was the only one of the big three (Danbooru, Sankaku) that worked with hydrus.

Attempting to open a video or a gif of duration > 0 in the media viewer directly causes hyrdus to hardlock with “100%” CPU usage (it actually only uses the majority of a single core). Attached is what the media viewer looks like after opening an affected file. Hyrdus doesn’t output any errors in the log so it is excluded. If a non-affected file is opened first and then the media viewer is scrolled to an affected file, hyrdus works as intended. If the affected files are set to open externally, hydrus works as intended. Regardless of settings, the preview works as intended. The checkboxes in the media options tab don’t seem to effect the bug.

Hey guys. The recent breaking of Gelbooru has given me reason to update Hydrus for the first time in ages (to version 254). Unfortunately, it now no longer starts. I updated by simply extracting the old files in the same directory as the new, using the "replace all" option at the prompt. System: Linux Mint Cinnamon 17.3 64-bit Some sort of Skylake i5 I forgot 12GB RAM When running it from the console, I got this feedback: Traceback (most recent call last): File "<string>", line 20, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientController", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientCaches", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDefaults", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientConstants", line 3, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx", line 49, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx._core", line 4, in <module> ImportError: No module named _core_ Apt policy tells me that the "python" package is installed (providing version 2.7.5) and the "python3" package is installed (providing version 3.4.0).

>>5705 It's pretty late my time, I'll check for replies tomorrow

>>5705 I just tried 255, same issue. console output: Traceback (most recent call last): File "<string>", line 20, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientController", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientCaches", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientDefaults", line 1, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientConstants", line 3, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx", line 49, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 276, in load_module File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/wx._core", line 4, in <module> File "/usr/local/lib/python2.7/dist-packages/PyInstaller-2.1.1dev_-py2.7.egg/PyInstaller/loader/pyi_importers.py", line 420, in load_module ImportError: /home/cfalcon/.hydrus/hydrus network/libX11.so.6: undefined symbol: xcb_poll_for_reply64

>>5734 What version were you updating from?

>>5119 It won't be a bug if it could be a feature! See >>5737 and >>5738 >>5129 is is safe to assume that 1920x1080 and 1366x768 are the top screen resolutions. Judging that most font require at most 16x16 (Latin and other western alphabet) or 32x32 (Logographic languages) per character, how much space does it take to display all tags for one image?

>>5736 It was pretty damn ancient. I checked the version just before I updated it and it was 170-something.

>>5743 Oh, here: >>5362 My first guess was a problem with the whole v245 thing.

>>5743 I am not the dev. Okay, do you have a backup? First thing to do is download the version you were on before and extract it over your current install. Then, launch it and see if it works. If it does, upgrade in steps of 5 versions difference. Launch the client after every upgrade. If it needs to do any kind of maintenance, let it do that maintenance, quit the client, and launch the client again. If it tells you you need an older version (eg you're upgrading to 234 and it tells you it needs v234), do that. Do this till you arrive at v255. If it does not, then IF AND ONLY IF you have a BACKUP, remove your install, install the version you used to have, and use it to restore the backup. But first check if that version is capable of importing a backup of course. In the gap between 170 and 255, hydrus has undergone several big changes (I don't know which) that cause it to handle everything differently, I remember the whole way files where saved being changed, so even if you could have launched the client it might not recognize your database.

>>5635 I probably either SIGTERM'd or SIGKILL'd it one of the times it appeared to be hanging when I tried to exit, I'll have to make sure to not do that. I'm running Void Linux, which is a rolling release distro like Arch but uses LibreSSL instead of OpenSSL. iirc v255 started downloading updates from the PTR then gave me the message that it had nuked the local state for it and I should restart hydrus while it was downloading 5 updates (as opposed to the hundreds i would get if it were starting from scratch.) After restarting hydrus it knows the PTR exists, but doesn't have any tag mappings from it.

>>5745 Thanks, I'll try to get source install to behave. It doesn't look like it's going to be easy, though - I know 0 python and some pip modules are failing with compile errors. Mint is based on Ubuntu (my version of Mint is specifically based on 14.04, and I can't update to Mint 18 because the LMDT FUBARed the GCC packages I absolutely need to do my work on), which is why the previous binary worked but maybe we're running different enough versions that the new one still fucked up. >>5746 Thanks, I'll try that if the new Hydrus doesn't read my DB correctly. First I'll need to get it started up though, it still doesn't work even when I extract the tarball into a new folder.

>>5676 >Honestly I'd recommend just not scraping from gelbooru (pretty much all of their content is mirrored from danbooru anyway) Problem is, hydrus does not scrape full-size images from danbooru properly.

Moving around in this webm spawns this error. UnboundLocalError local variable 'g' referenced before assignment File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 862, in EventPaint File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientGUICanvas", line 734, in _Redraw File "/home/hydrus/Desktop/hydrus/build/client/out00-PYZ.pyz/include.ClientData", line 235, in GetDifferentLighterDarkerColour I've seen it around other webms, but this one makes it pretty reliable.

>>5771 use tbib?

>>5678 Danbooru does have loli hid behind a paywall TECHNICALLY, there are workarounds BetterBooruBrowser works for your browser. And I guess we're probably not interested in the same stuff as most things I'm looking for are mirrored on danbooru and gelbooru. >>5679 >>5745 Ah, I forgot that most of the downloaders in hydrus are broken in some way. I've been using my own home-baked scraper that exports to image+txt file tag lists for a while now and didn't even think about that. >>5773 This might be a viable option if you want to use hydrus to pull your delicious loli. I don't know if hydrus is configurable to work with tbib though. https://ibsearch.xxx/ might also be an option? I'd make some configs and share but my hydrus is bork a.t.m. :(

>>5777 And as to my hydrus being bork, here's a traceback: 2017/05/14 15:39:01: booting controller…
2017/05/14 15:39:01: booting db…
2017/05/14 15:39:01: preparing disk cache
2017/05/14 15:39:02: preparing db caches
2017/05/14 15:39:02: A serious error occured while trying to start the program. Its traceback will be shown next. It should have also been written to client.log.
2017/05/14 15:39:02: Traceback (most recent call last):
File "C:\Hydrus Network\include\ClientController.py", line 1149, in THREADBootEverything
self.InitModel()
File "C:\Hydrus Network\include\ClientController.py", line 574, in InitModel
self._client_session_manager = ClientCaches.HydrusSessionManager( self )
File "C:\Hydrus Network\include\ClientCaches.py", line 1528, in __init__
existing_sessions = self._controller.Read( 'hydrus_sessions' )
File "C:\Hydrus Network\include\HydrusController.py", line 285, in Read
return self._Read( action, *args, **kwargs )
File "C:\Hydrus Network\include\HydrusController.py", line 95, in _Read
result = self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
File "C:\Hydrus Network\include\HydrusDB.py", line 783, in Read
return job.GetResult()
File "C:\Hydrus Network\include\HydrusData.py", line 1662, in GetResult
raise e
DBException: TypeError: 'NoneType' object is not iterable

Database Traceback (most recent call last):
File "C:\Hydrus Network\include\HydrusDB.py", line 516, in _ProcessJob
if job_type in ( 'read', 'read_write' ): result = self._Read( action, *args, **kwargs )
File "C:\Hydrus Network\include\ClientDB.py", line 7287, in _Read
elif action == 'hydrus_sessions': result = self._GetHydrusSessions( *args, **kwargs )
File "C:\Hydrus Network\include\ClientDB.py", line 4384, in _GetHydrusSessions
service = self._GetService( service_id )
File "C:\Hydrus Network\include\ClientDB.py", line 5066, in _GetService
( service_key, service_type, name, dictionary_string ) = self._c.execute( 'SELECT service_key, service_type, name, dictionary_string FROM services WHERE service_id = ?;', ( service_id, ) ).fetchone()
TypeError: 'NoneType' object is not iterable This is on version 254, Windows 8.1 x64. This happened after booting hydrus, in my previous session I had imported a load of tags from img+txt bundles into a remote tag repo realized my scraper was mangling unicode, so I needed to delete all the mappings which the advanced tag operations dialog had no options for on remote tag repos, so I deleted the entire service from my server and client and re-added a new one. It appeared to have worked fine until I reopened my client and got an error dialog. It seems like my last session (which is set to load on startup) or some metadata is referencing the deleted service? I'm going to muck with the database to see if I can get hydrus to boot, but figured I'd report it so no one else gets hit by this.

>>5778 Sorry for posting three times in a row :/. But I fixed my hydrus by mucking with my database, there were still references to the deleted service in the main client.db in the tables hydrus_session and tag_siblings. Deleting the entries referencing service_id 27 (which I guessed was the deleted repo as it was missing in the services table) made hydrus boot. Hopefully there is nothing else that is lingering around still referencing the deleted service.

Alright, this is on v 255, on arch linux. Basically, I can't connect to hydrus tag server. Traceback:
NetworkException
Could not connect to hydrus.no-ip.org:
Traceback (most recent call last):
File "/opt/hydrus/include/HydrusThreading.py", line 276, in run
callable( *args, **kwargs )
File "/opt/hydrus/include/ClientGUI.py", line 2788, in _THREADUploadPending
service.Request( HC.POST, 'update', { 'client_to_server_update' : client_to_server_update } )
File "/opt/hydrus/include/ClientServices.py", line 722, in Request
( response, size_of_response, response_headers, cookies ) = HG.client_controller.DoHTTP( method, url, request_headers, body, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = True )
File "/opt/hydrus/include/ClientController.py", line 388, in DoHTTP
def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
File "/opt/hydrus/include/ClientNetworking.py", line 357, in Request
( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 268, in _DoRequest
connection = self._GetConnection( location, hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 340, in _GetConnection
connection = HTTPConnection( location, hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 422, in __init__
self._RefreshConnection()
File "/opt/hydrus/include/ClientNetworking.py", line 885, in _RefreshConnection
raise HydrusExceptions.NetworkException( text )
NetworkException: Could not connect to hydrus.no-ip.org:

('_ssl.c:645: The handshake operation timed out',)


('_ssl.c:645: The handshake operation timed out',)


>>5781 Same shit here.

>>5781 Anon should update/fix their ssl probably the dns works fine, doesn't return ping but I don't expect it to

>>5786 Hmm. So this could be related to the new openssl update couple weeks back? Hydrus give no problem after that, so I thought nothing is wrong. Lemme try changing the environment variable… Okay, so I run hydrus with this and still got the problem: LD_LIBRARY_PATH=/usr/lib/openssl-1.0-compat/ hydrus-client Anyway, traceback. I think it's similar?
NetworkException
Could not connect to hydrus.no-ip.org:
Traceback (most recent call last):
File "/opt/hydrus/include/HydrusThreading.py", line 276, in run
callable( *args, **kwargs )
File "/opt/hydrus/include/ClientGUI.py", line 2788, in _THREADUploadPending
service.Request( HC.POST, 'update', { 'client_to_server_update' : client_to_server_update } )
File "/opt/hydrus/include/ClientServices.py", line 722, in Request
( response, size_of_response, response_headers, cookies ) = HG.client_controller.DoHTTP( method, url, request_headers, body, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = True )
File "/opt/hydrus/include/ClientController.py", line 388, in DoHTTP
def DoHTTP( self, *args, **kwargs ): return self._http.Request( *args, **kwargs )
File "/opt/hydrus/include/ClientNetworking.py", line 357, in Request
( response, size_of_response, response_headers, cookies ) = self._DoRequest( method, location, path, query, request_headers, body, follow_redirects = follow_redirects, report_hooks = report_hooks, temp_path = temp_path, hydrus_network = hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 268, in _DoRequest
connection = self._GetConnection( location, hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 340, in _GetConnection
connection = HTTPConnection( location, hydrus_network )
File "/opt/hydrus/include/ClientNetworking.py", line 422, in __init__
self._RefreshConnection()
File "/opt/hydrus/include/ClientNetworking.py", line 885, in _RefreshConnection
raise HydrusExceptions.NetworkException( text )
NetworkException: Could not connect to hydrus.no-ip.org:

('_ssl.c:645: The handshake operation timed out',)


('_ssl.c:645: The handshake operation timed out',)


Sorry lads, I kept on putting answering this thread off, and now there's all the more to catch up on. >>5637 >>5656 I am not sure if there is something specifically going wrong here or just a bad luck combination of my ugly code and the way your machine happens to start and stop the maintenance. Please set your big idle jobs to only run on shutdown (on that same options panel) for a while and see how that works for you. I am also chasing down the causes of the occasional memory bloat. I have some ideas on what is doing it and hope to roll out some improvements on that front in the next few weeks. >>5661 >>5664 >>5669 >>5671 >>5675 >>5676 >>5677 >>5678 >>5679 I've purged these shit redirect urls for tomorrow's release, and the gelbooru parser will now chase down these redirects to the proper urls, so it should be working as it once was. Feel free to refresh your affected subs again to catch up. >>5662 That's odd–I assume those two files are the exact same thing, you just copied and renamed? I can't reproduce this problem, even with some complicated kana thrown in. Which version of Windows is that in your screenshot, 7? 8? Everything is explicitly converted to unicode inside hydrus, but I wonder if they are still dealing with it on their end in a different way. Is there any difference if you drop the unicode file on the main gui window vs dropping it on that import dialog? >>5700 I may have fixed this last week, as another user had a similar issue. If you still get the problem, what are your settings for 'media_viewer' under options->gui? Setting 'remember position/size' to True also fixed the issue. >>5756 I am sorry for the inconvenience here. I didn't think this was essentially possible, but I think spreading sqlite transactions across multiple db files isn't as atomic as I'd hoped. >>5772 Thank you for this report. I made a typo, and my typo-checker didn't catch it. This is fixed for tomorrow. >>5778 >>5780 Thank you for this excellent report. I am sorry for the inconvenience. I will look at this more closely tomorrow and make a plan to makke sure everything is being deleted correctly and that nothing can sneak in afterwards. >>5781 >>5785 >>5786 >>5788 I dunno what happened here. I restarted my server and it worked again. I updated some openssl stuff recently and updated my server last week, so maybe that library got borked somehow. At the moment I am scratching this up to something out of my control getting stuck due to bad luck, but if it happens again, I'll look into it more closely.

[Expand Post]

>>5757 I am sorry I missed this. The advice already given is good. I don't know a great deal about Linux, but getting it all going from source with pip can be a hassle. Since you are coming from such an old version, even when you get it booting, it'll may tell you I can't update this much in one go, so at some point you'll might have to rewind. If you are at v170, I recommend trying v210. If you can avoid it, try to skip v195-v205. They were a mess. Or you can go five or ten versions at a time.

>>5808 Network issue is indeed fixed now. Thanks, dev!

>>5756 >>5808 After I let hydrus (v255) get the updates it threw a traceback(again) immediately when it started processing, manually deleting the repo and re-adding it appears to fix it. traceback:
DBException
DataMissing: Service tag map error in database
Traceback (most recent call last):
File "/home/username/git/hydrus/include/ClientServices.py", line 1282, in SyncProcessUpdates
( did_some_work, did_everything ) = HG.client_controller.WriteSynchronous( 'process_repository', self._service_key, only_when_idle, stop_time )
File "/home/username/git/hydrus/include/HydrusController.py", line 386, in WriteSynchronous
return self._Write( action, HC.LOW_PRIORITY, True, *args, **kwargs )
File "/home/username/git/hydrus/include/HydrusController.py", line 117, in _Write
result = self._db.Write( action, priority, synchronous, *args, **kwargs )
File "/home/username/git/hydrus/include/HydrusDB.py", line 824, in Write
if synchronous: return job.GetResult()
File "/home/username/git/hydrus/include/HydrusData.py", line 1668, in GetResult
raise e
DBException: DataMissing: Service tag map error in database
Database Traceback (most recent call last):
File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 9493, in _Write
elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 7249, in _ProcessRepositoryUpdates
did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
File "/home/username/git/hydrus/include/ClientDB.py", line 6240, in _ProcessRepositoryContentUpdate
tag_id = self._CacheRepositoryNormaliseServiceTagId( service_id, service_tag_id )
File "/home/username/git/hydrus/include/ClientDB.py", line 697, in _CacheRepositoryNormaliseServiceTagId
raise HydrusExceptions.DataMissing( 'Service tag map error in database' )
DataMissing: Service tag map error in database


Database Traceback (most recent call last):
File "/home/username/git/hydrus/include/HydrusDB.py", line 517, in _ProcessJob
elif job_type in ( 'write' ): result = self._Write( action, *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 9493, in _Write
elif action == 'process_repository': result = self._ProcessRepositoryUpdates( *args, **kwargs )
File "/home/username/git/hydrus/include/ClientDB.py", line 7249, in _ProcessRepositoryUpdates
did_whole_update = self._ProcessRepositoryContentUpdate( job_key, service_id, content_update )
File "/home/username/git/hydrus/include/ClientDB.py", line 6240, in _ProcessRepositoryContentUpdate
tag_id = self._CacheRepositoryNormaliseServiceTagId( service_id, service_tag_id )
File "/home/username/git/hydrus/include/ClientDB.py", line 697, in _CacheRepositoryNormaliseServiceTagId
raise HydrusExceptions.DataMissing( 'Service tag map error in database' )
DataMissing: Service tag map error in database


>>5811 I found the bug that made me pkill hydrus, I paused the repo processing and told hydrus to exit late last night and it's been hanging since then. Here's the last few lines in the terminal: 2017/05/17 02:01:53: processed 30,662,109 content rows at 5,417 rows/s
2017/05/17 02:08:10:
2017/05/17 02:08:10: Exception:
2017/05/17 02:08:10: PyDeadObjectError: The C++ part of the _ServiceTagPanel object has been deleted, attribute access no longer allowed.
File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
self.__bootstrap_inner()
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/username/git/hydrus/include/HydrusThreading.py", line 286, in run
HydrusData.ShowException( e )
File "/home/username/git/hydrus/include/HydrusData.py", line 858, in PrintException
trace_list = traceback.format_stack()
2017/05/17 02:08:10:
2017/05/17 02:08:10: Exception:
2017/05/17 02:08:10: PyDeadObjectError: The C++ part of the _ServiceTagPanel object has been deleted, attribute access no longer allowed.
File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
self.__bootstrap_inner()
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/username/git/hydrus/include/HydrusThreading.py", line 286, in run
HydrusData.ShowException( e )
File "/home/username/git/hydrus/include/HydrusData.py", line 858, in PrintException
trace_list = traceback.format_stack()
2017/05/17 02:08:12:
2017/05/17 02:08:12: Exception:
2017/05/17 02:08:12: PyDeadObjectError: The C++ part of the _ServiceRepositoryPanel object has been deleted, attribute access no longer allowed.
File "/usr/lib/python2.7/threading.py", line 774, in __bootstrap
self.__bootstrap_inner()
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/home/username/git/hydrus/include/HydrusThreading.py", line 286, in run
HydrusData.ShowException( e )
File "/home/username/git/hydrus/include/HydrusData.py", line 858, in PrintException
trace_list = traceback.format_stack()
2017/05/17 02:08:12: PTR sync: processing updates


>>5808 >>>5662 > >That's odd–I assume those two files are the exact same thing, you just copied and renamed? I can't reproduce this problem, even with some complicated kana thrown in. Which version of Windows is that in your screenshot, 7? 8? Everything is explicitly converted to unicode inside hydrus, but I wonder if they are still dealing with it on their end in a different way. Is there any difference if you drop the unicode file on the main gui window vs dropping it on that import dialog? Yep both files are the exact same thing just copied and renamed. file names are: Jessica Pare - Stardom (2000) GracefulSickIberianbarbel.webm Jessica Paré - Stardom (2000) GracefulSickIberianbarbel.webm I'm running Windows 7 SP1 x64. I've got Python v2.7.10 installed (and on %path%) that I've been using for my own small programs. Testing import methods with Hydrus v256 extractonly: file>import files->add folder button->select directory 1 successful, 1 unsupported drag drop both files onto main client window 1 successful, 1 unsupported drap drop both files onto file->import files window 1 successful, 1 unsupported

I am unable to type or paste into the thread watcher URL field on v256 using Linux Mint 18. It seems like this problem is only on the thread watcher.

>>5872 See >>5874

>>5827 Thank you for this update and the exact filenames. After exploring the issue more, i was able to reproduce it and write a fix that seems to work. Please let me know if you get this again after v257! >>5872 I apologise for this! It is fixed for v257. If it is very important to you, it is safe to roll back to v255 this week. >>5822 Thank you for these tracebacks. I think I have them all cleaned up for v257.

So just how big is 368271232 pixels? I got a second decompression bomb warning in the past two days, took 10 minutes for my computer to react to ctrl alt del, then said it couldn't load the security shit, then another few minutes to just open the task manager, which showed hydrus using just under 2gb of ram (not bad) but the commit size was 12gb. I only vaguely understand what that means, and that it's virtual memory, but I've only got the 8gb of physical memory. Is there a better way to deal with these? Or a way to have actual information on what the file was in the log?

>>5948 >open back up >go back to the search I was on >there actually was a 37MB, 14,173x25,984 image grabbed from gelbooru Well, fuck.

This traceback happens when you attempt to do a "one-time import or delete using a hydrus tag archive" from advanced service-wide update in remote review services. using version 257. TypeError
object of type 'builtin_function_or_method' has no len()
File "C:\Hydrus Network\include\ClientGUICommon.py", line 262, in EventButton
self._func( *self._args, **self._kwargs )
File "C:\Hydrus Network\include\ClientGUIScrolledPanelsReview.py", line 272, in ImportFromHTA
ClientTags.ImportFromHTA( self, path, self._service_key, self._hashes )
File "C:\Hydrus Network\include\ClientTags.py", line 75, in ImportFromHTA
hash_type = hta.GetHashType() # this tests if the hta can produce a hashtype
File "C:\Hydrus Network\include\HydrusTagArchive.py", line 206, in GetHashType
if len( hash ) == 16: self.SetHashType( HASH_TYPE_MD5 ) Looks like you renamed a local variable "hash" to "result" at some point and forgot to rename all it's usages.

>>5948 >>5949 I am sorry about this–I don't really know any easy solutions to it. Most of this stuff happens at a lower level than I usually work, so for instance I typically don't know a file is gigantic until it has been fully loaded. Your 370Mpixel image takes up about 1GB of memory (pixels*3bytes for RGB, or 4 bytes if it is a png with transparency). But I presume Pillow or OpenCV–whichever is opening the file for you–needs a bit of extra space to write everything to that final memory canvas. 12GB seems waaaay excessive for that, but I don't know the fine details. I cleaned up some of my side of that a couple of weeks ago, but perhaps there is some additional doubling going on, either in the C->Python step or in my own code. I guess some part of it all triggers the decompressionbomb warning, although in this case even the warning is a false positive, as I presume the file is legit. I figure we'll all be running with 64GB of ram in a few years, and worrying about 100kx100k textures bloating out our 2TB game downloads. Do you have a link to that file, so I can test it my end? >>5964 Damn, thank you! I will make sure to have this fixed for next week.

>>5979 https://gelbooru.com/index.php?page=post&s=view&id=444727 Didn't think there would be anything that big, especially not when grabbing everything from a series that began and fully ended pre-pixiv.

>>5985 The next largest files I've come across were a series of small poster scans, like the kind roughly a sheet of paper's width across, rolled up in a little blind packaged box. 2,145x6,075, png, 20 to 25 mb each and it's just stupid. I don't go around downloading huge single image files for fun or something.

hydrus (v258) threw a traceback on exit but nothing bad appears to have happened: 2017/06/04 03:23:39: waiting for daemons to exit
2017/06/04 03:23:39: Traceback (most recent call last):
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/ClientGUICommon.py", line 3812, in EventDirty
2017/06/04 03:23:39: 2017/06/04 03:23:39: self._func( *self._args, **self._kwargs )
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/ClientGUI.py", line 3406, in RefreshMenu
2017/06/04 03:23:39: 2017/06/04 03:23:39: ( menu, label, show ) = self._GenerateMenuInfo( name )
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/ClientGUI.py", line 1550, in _GenerateMenuInfo
2017/06/04 03:23:39: 2017/06/04 03:23:39: elif name == 'pages': return pages()
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/ClientGUI.py", line 1102, in pages
2017/06/04 03:23:39: 2017/06/04 03:23:39: gui_session_names = self._controller.Read( 'serialisable_names', HydrusSerialisable.SERIALISABLE_TYPE_GUI_SESSION )
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/HydrusController.py", line 290, in Read
2017/06/04 03:23:39: 2017/06/04 03:23:39: return self._Read( action, *args, **kwargs )
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/HydrusController.py", line 95, in _Read
2017/06/04 03:23:39: 2017/06/04 03:23:39: result = self._db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
2017/06/04 03:23:39: File "/home/runningdroid/git/hydrus/include/HydrusDB.py", line 813, in Read
2017/06/04 03:23:39: 2017/06/04 03:23:39: raise HydrusExceptions.ShutdownException( 'Application has shut down!' )
2017/06/04 03:23:39: include.HydrusExceptions2017/06/04 03:23:39: .2017/06/04 03:23:39: ShutdownException2017/06/04 03:23:39: : 2017/06/04 03:23:39: Application has shut down!

Slideshow isn't working for some reason.

hydrus does not ask if I want to run maintenance jobs on shutdown and does not run them.

>>6082 Thank you for this report. This should be harmless (these are usually silenced)–it just looks like it got reported and printed by accident. I will track it down and fix it. >>6089 Thank you for this report. This should be fixed in v259! Let me know if you still have any problems. >>6116 I recently reduced the threshold for this test. It is less likely to ask you when there is only a small amount of work to do–it'll batch it up for later. I will be adding options to control this behaviour in future!

hydrus version 258, happened when trying to add four tags in the same namespace, where three of the four where added with the keyboard hold ctrl+arrow keys, and one by scrolling down and CTRL clicking. three of the four where not on screen when trying to add by pressing enter.
SizeException
Received a zero-length tag!
Traceback (most recent call last):
File "include\HydrusDB.py", line 534, in _ProcessJob
result = self._Write( action, *args, **kwargs )
File "include\ClientDB.py", line 9896, in _Write
elif action == 'push_recent_tags': result = self._PushRecentTags( *args, **kwargs )
File "include\ClientDB.py", line 7659, in _PushRecentTags
tag_ids = [ self._GetTagId( tag ) for tag in tags ]
File "include\ClientDB.py", line 5743, in _GetTagId
HydrusTags.CheckTagNotEmpty( tag )
File "include\HydrusTags.py", line 150, in CheckTagNotEmpty
raise HydrusExceptions.SizeException( 'Received a zero-length tag!' )
SizeException: Received a zero-length tag!



>>6134 I should add, all the four tags did get added, and one extra tag in the namespace, which looks like it doesn't contain anything has been added

>>6134 >>6135 Thank you for this report and traceback. I will look at it next week!

If you petition a non-local tag on an item, that item will still show up when you search the tag you petitioned away from it until you actually commit the petition to the repo in question.

I don't know if it's a bug or working as intended, but I was checking for duplicates manually (checking artist tags and similar images from the menu, each opening a new tab) and after ~200 images I noticed my system slowing down big time. Hydrus was hoarding over 7GB of ram according to task manager.

>>6144 Thanks–yeah, unfortunately, the logic behind treating un-committed deletes as real deletes gets more complicated than I can currently maintain. For now, the file has both the 'current' and 'petitioned' state, although different elements across the program have differing abilities to understand that based on how sophisticated they are. Please commit your pending and petitioned when you are confident they are good and do not rely on accurate counts and so on until the change (and all the related computation to confirm the new 'deleted' status) is done. >>6152 Thank you for this report. Several users have reported memory explosion, and I have experienced it once myself. I am not sure what is doing it, but I have some suspicions. Please keep a continuing eye open for this and note down anything that you might thing contributes to it. I'm particularly interested in knowing– How many thumbnails, approx, were open across the program. How many tabs were open. Whether an import queue (even a silent one like an import folder) was happening, and what sort of files it was importing. How long the client had been open. Whether you had been browsing many images or videos recently. If you remember any answers to those from this recent time, please let me know.

For a couple of months now, the client has been crashing on and off for me when opening the "add sibling to tag" menu from the manage-tag dialog with a tag highlighted, I'm currently on win64 v258. Hydrus doesn't just hang, Windows almost immediately throws the "hydrus.exe has stopped working" box in my face, Hydrus doesn't write anything to any of the logs kept, not even a crash.log.

>>6206 Thank you for this report. I am afraid I cannot reproduce it on Win10. That dialog does take like 5 seconds to load now and is otherwise a big mess behind the scenes that can't deal with the current mass of pairs, so I can see how this could cause problems. Do you get the crash if you open the dialog straight from services->manage tag siblings? Are you asking to set siblings on tags that do or do not already have siblings? I'll see about shifting the initialisation of the dialog around so it doesn't hang everything before it even exists. That sort of out-of-order event blocking tends to be the only thing that can cause a full gui crash for hydrus.

pixiv urls are http instead of https

>>6194 It's been a long time but I'll try to remember >How many thumbnails, approx, were open across the program. I'm assuming you mean how many images I had on tabs across the program, at one time probably ~2000 but in total during that session ~3000. Most of those were on tabs I didn't actually open that session so maybe they don't count for this? If that's the case, then it's less than a 1000. >How many tabs were open. 7-10, maybe? >Whether an import queue (even a silent one like an import folder) was happening, and what sort of files it was importing. Nothing that left any notifications was happening >How long the client had been open. Like 6 hours or so. >Whether you had been browsing many images or videos recently. Nothing apart from the duplicate checking

>>6211 I just tried it out again, I loaded "manage tag siblings" a couple of times, one time opening an image in between. When I subsequently loaded another image and used "add siblings to *tag*" it immediately crashed. After a couple of control runs it doesn't seem to matter whether the tag's already pending, what kind of tag I'm opening/what kind of namespace it's in or whether it already has siblings or not, the first couple of times the dialog is launched seem to succeed, and the crashes usually happen when I've opened the dialog for at least one *different* tag before. Other bugs and circumstances I'm experiencing that I think might also be of note: - I'm currently also having extreme slowdown issues with the tag autocompletion. Typing the first couple of letters in the tag: field when managing tags takes 10-60 seconds per symbol, while I'm using a Velociraptor disk for my DB, which is no SSD but should be fairly fast. Regenerating the cache doesn't seem to help all that much. This may be related to me running multiple tag repo's on a local server process to seperate collected tags from different sources and keep them from 'contaminating' the PTR when they're shitty (fucking tumblr blogs). - I also have ~500k tags pending for the PTR because I pended the 'title:' tags from the partial FurAffinity dump. And for more of a suggestion than a bug: I know you warned about trying out the Tumblr '_raw' URLs, but I think you might want to add an option to the downloader to it make fall back to trying out a non-raw version of a URL when downloading fails on a '_raw' one. It's currently failing the entire synchronization of some of my new subscriptions on the 403's Tumblr occasionally hands out. http://cremechouette.tumblr.com/ is an example case that currently fails for me.

Now sure why, but I get some weirdness on closing hydrus in win 7 64 bit. It closes properly, the client.exe task clears from task manager but my system basically locks up for a minute or two afterwards. HDD light either stays on or flashes on/off for half a second each throughout the whole minute or two. It doesn't effect anything currently going i.e. sound doesn't lag or lock up but I can't do anything besides move the mouse around the desktop.

>>6226 It's probably hydrus calling the Windows equivalent of fsync() on the databases after having filled the cache with writes. I don't have access to a Windows box to test but you might be able to keep it from locking up your desktop by reducing hydrus' priority, I tend to set hydrus' io priority to idle when i'm doing something else and it seems to keep everything working fine.

When using the pixiv artist id downloader, the first page of images will get parsed properly but any pages after that report "0 urls found".


PyAssertionError
C++ assertion "(itemid >= 0 && itemid < SHRT_MAX) || (itemid >= wxID_AUTO_LOWEST && itemid <= wxID_AUTO_HIGHEST)" failed at ../src/common/menucmn.cpp(260) in wxMenuItemBase(): invalid itemid value
File "/home/user/.git/hydrus/include/ClientGUIListBoxes.py", line 861, in EventMouseRightClick
menu.Append( ClientCaches.MENU_EVENT_ID_TO_ACTION_CACHE.GetTemporaryId( 'new_search_page' ), 'open a new search page for ' + selection_string )
File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/_core.py", line 12003, in Append
return _core_.Menu_Append(*args, **kwargs)
got this error when right clicking on search result 'avengers' on ptr

I am sorry for the late replies. IRL has been hammering me just recently. >>6213 This should be fixed now! All new pixiv urls are https and all existing ones should be converted on update! >>6215 I have heard this problem is relieved recently, after some recent memory maintenance and other internal cleanup stuff I have been working on. Please let me know if you are still running into trouble. >>6216 Thank you for this update. I am talking with several other people about similar issues, and I even got a couple of crashes myself doing similar things. I am putting a little time into this every week. I can't remember if you were testing this before I changed how tag siblings loads, but I recently put it on a thread and have had no problems beyond a bit of lag. I would like still like to rewrite the whole thing to fix the whole mess, but I think I've fixed the crash. The tumblr issue is now also fixed, I think! The new raw urls sometimes need the 68. part of their domain clipped, which now happens in hydrus automatically. That cremechouette works ok for me now. >>6226 >>6243 This is interesting–I haven't heard of this before. I don't know enough about Win 7 to comment, but if you reduce your maximum shutdown idle time to like 3 minutes, then there won't be as many pending writes. If this is happening without any shutdown tag processing going on, then I don't know what is going on. The db resets its connection to clear its journal (buffer, essentially) every like 30 minutes or something, so it shouldn't be able to build up a huge amount to commit without a load of actual recent work going on. >>6251 Thank you for this report. I will check it out this week. Do you have an example query I can look at, just in case my tests go ok? >>6267 This is the menu id overflow bug, which I am slowly fixing by replacing all the old menu code with a newer system. The fix for now is to restart the client. Please let me know if you are able to reproduce this soon after starting the client.

>>6226 >>6274 I haven't tested it too extensively but I know for sure that it happens after vacuuming the db. It doesn't happen if I pop open hydrus for a small task and close but it does if I do an import of hundreds of images or do dupe processing and vacuum the db afterwards. (I manually do this regularly without issue and I don't think I've ever waited for a scheduled vacuum) It says maintenance is complete before I close it so it's not like I'm cutting it off in the middle but if client.exe is closed I'm not sure any sort of db logs could catch what's going on.

>>6274 >This is the menu id overflow bug, which I am slowly fixing by replacing all the old menu code with a newer system. The fix for now is to restart the client. Please let me know if you are able to reproduce this soon after starting the client. unfortunately i can't one thing i can remember is that there is tag with many character, and the error happened. after that right click will produce the error until application is restarted.

I'm guessing I hit a limit rather than a bug. I'll admit I haven't updated in a while (client 256).

>>6374 It's been fixed, update.

Gelbooru page downloads stopped working. What is happening is the first image becomes the URL for all the images so only 1 image is downloaded (see pic).

A lil' bug report 1. Hydrus will silently crash if the custom thumbnail folder cannot be found. This is a pretty huge issue when migrating a DB to another installation. Possible solution would be to display a small dialog saying "could not find thumbnail folder", with options to abort startup or to unset old thumbnail folder. 2. On my linux installation (lubuntu, 1280 x 800 screen), most of the small dialog boxes (rating, add or remove tag, commit/confirm, etc) are slightly to short, and require dragging every time. I'm pretty sure WxPython has an option to wrap contents in this case.

>>6374 >>6375 There is an additional fix for gelbooru in the release I will put out later today, so make sure you get v266! They've been all over the map just recently. Let me know if you still get any problems like this in the latest version. >>6383 Thank you for this report. It should be fixed in today's release–let me know if it isn't. >>6387 Thank you for these reports. I will check on the custom thumbnail folder thing. I am not sure why it would cause a crash and not an error–was this on client boot, or did the folder dismount/whatever while the program was up? That will have better ui in today's release, btw. I'll be having a look at Linux layout as well next week. Someone else emailed with similar. I'll see if I can fix the sizing calculation, or just add padding if not.

If you make a shortcut for a tag that is shown as another tag, like 1girl, when you activate the shortcut, it will add the tag you specified to the image you're on, but the shortcut won't remove that tag. It only works in removing the tag if you specify a tag that won't be replaced.

>>6462 Thank you for this report. I will try to look at this tomorrow.

>>6395 >I will check on the custom thumbnail folder thing. I am not sure why it would cause a crash and not an error–was this on client boot, or did the folder dismount/whatever while the program was up? It was on client boot.

Don't know if this is a bug or not, but everytime I run a DB vacuum(soft or full), I can never tell if it's actually working or not. I don't get any type of message.

>>6539 It might be that it is not running due to limited free space on your system partition/temporary folder. I think there are a couple of other checks it does as well. I will make a job to better inform the user when a manual attempt fails for this sort of unusual reason.

Whenever I try to drag and drop an image into discord from hydrus the 'drop image here' window pops up but the cursor turns into this symbol and doesn't allow me to upload the image. Every other service like 8ch/4ch/any image host in chrome works just fine but discord and hydrus just seem to not want to cooperate. Is this fixable? This is the one and only problem I have ever had with hydrus.

>>6591 As far as I know, this is a discord problem. Apparently some other media/IM programs have a similar problem dragging files to either the Discord app or web page. I understand they privilege (increase the security level of) their program in some way, and so hydrus, which doesn't ask for any special permission from Windows, isn't allowed to drag and drop to it. You can get similar problems trying to DnD between a program that is run normally vs a program run explicitly as admin. In any case, all I do my end is say 'here are some filenames for a DnD event' in the normal OS-appropriate way, and then Windows takes over. The 'no way' symbol is something to do with Discord saying 'I won't accept that' or Windows saying 'I can't trust that'. Solutions at the moment are to DnD to an interim Windows explorer window (or your desktop) and then to Discord, or to hit Ctrl+C on your hydrus thumbnail selection (copying the file paths to your clipboard) and then paste them in the file selection dialog (launched by hitting the little up arrow on the side of the Discord text entry box).

>>6609 I will use these methods in the future. Thanks for all your hard work dev-sama

I recently updated to 270, and whenever I open it, it crashed at startup telling me that module 'cv2' doesn't have the 'CV_LOAD_IMAGE_UNCHANGED' field I run from source, with dependencies installed separately What can I do to fix it?

>>6622 I am sorry you are having this problem. I am not sure what is going on here. I juggled some of the opencv constants a little in v270, but that one remains the same as in v269. Was v269 the last version you were running, or a bit before? Can you please try downloading the source for the last version you had running to a new location and seeing if a fresh client will boot? Also, please open a terminal and go: python
import cv2
cv2.__version__
cv2.CV_LOAD_IMAGE_UNCHANGED
cv2.IMREAD_UNCHANGED
exit() What version does it think it is, and which of the constants failed? If v269 was your last hydrus version, I believe that you can downgrade this week. If you try this, please let me know if it works, as this will show whether hydrus has changed or your opencv installation.

>>6630 My last installed version was v268, because I had forgot about the new release I jumped to v270 from there, skipping a version For a moment I thought it was because of python (I have both 2 and 3 installed), but clinent.py correctly calls python2, and python3 can't even find the cv2 module Running the commands, these are the value I get: cv2.version => 2.4.13 cv2.CV_LOAD_IMAGE_UNCHANGED => -1 cv2.IMREAD_UNCHANGED => -1L The issue now is that trying to run hydrus again tells me the issue is on cv2.CV_LOAD_IMAGE_ANYDEPTH

>>6631 Not sure if it helps, but I just thought writing down the OS might be better I run hydrus on GNU/Linux, as I said from source The dependencies are installed through the package manager and are automatically updated when I update the system So far I didn't have any problem though, this is the first time something like this happens

>>6632 >>6631 Thank you for this update. I have a temporary patch in place for tomorrow's release, so you should be able to boot. Some of the constant names on the old version of OpenCV are unusual and badly documented, and I do not have a testing environment for it here. Please try this: python
import cv2
cv2.CV_LOAD_IMAGE_ANYDEPTH
cv2.cv.CV_LOAD_IMAGE_ANYDEPTH
cv2.CV_LOAD_IMAGE_ANYCOLOR
cv2.cv.CV_LOAD_IMAGE_ANYCOLOR
exit() If you would like to update to the new (3.3.0) version of OpenCV manually, you can with: pip install opencv-python The new version is great and has all kinds of GPU hardware acceleration for rendering, but if you have more programs that use 2.X, you may want to hold off unless you know they are 3.X compatible. If your package manager has an option to update it, that might also be the better option.

>>6638 I'll just paste the console output: >>> import cv2
>>> cv2.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYDEPTH'
>>> cv2.cv.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYDEPTH'
>>> cv2.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'
>>> cv2.cv.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'
>>> exit() Apparently, hydrus is the only program using opencv, I upgraded it the last available through the PM, which is 3.2.0 Unfortunately the situation didn't change: >>> import cv2
>>> cv2.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYDEPTH'
>>> cv2.cv.CV_LOAD_IMAGE_ANYDEPTH
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'cv'
>>> cv2.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'CV_LOAD_IMAGE_ANYCOLOR'
>>> cv2.cv.CV_LOAD_IMAGE_ANYCOLOR
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'cv'
>>> exit()

>>6641 Nevermind, upgrading to 3.2.0 fixed the issue Which makes sense, poking at the code, I see that the affected line happens only when OpenCV is at version 2 Still, you might look at it for those with version 2 Thanks for your help and keep the hard work

Opening webm files externally on Fedora 26 open up Notepad in Wine instead of mpv. Outside of Hydrus, all webms are set to open with mpv by default. This makes Hydrus useless for me, because I specifically got it to manage porn webms.

>>6642 >>6641 I am glad this got fixed. I have a patch in place for the 2.X users that just won't support some of the new EXIF rotation detection stuff. I will dive into the docs a bit more if there is demand but otherwise leave it alone. >>6660 At the moment, for Linux, I run: xdg-open "[file_path]" I'm by no means an expert on this stuff, but could your xdg webm mapping for webm be unusual? What happens if you try that command in a regular terminal on an exported webm? What happens if you try to open a jpeg or an mp4 externally in hydrus? Do they go to the right place? I would like to add options for custom launch paths on a per mime basis in the future, but for the moment, I just pass it off to the OS to figure out.

>What happens if you try that command in a regular terminal on an exported webm? Using xdg-oen on regular webms opened mpv. Exported webm from hydrus, used xdg-open to open it, still loaded mpv. >What happens if you try to open a jpeg or an mp4 externally in hydrus? Do they go to the right place? Jpegs were set to show in the in-built viewer by default, which worked fine. After changing them to be opened externally, it opened it up in Wine Internet Explorer instead of my image viewer. Also used xdg-open on jpegs, worked as expected, opening the image viewer. Mp4, when set to open externally, opened up Wine Notepad. This is pretty weird.

>>6673 After some searching, I found that sometimes wine notepad is set for unknown mimetypes. I wonder if the hydrus terminal is giving file info in a different way, or looking up the association in a different way. Is there any way that hydrus is running as a different user with different lookup table? I'll reiterate that I'm not good at this, so please correct me if you know better, but please go: xdg-mime query filetype [webmpath]
xdg-mime query default video/webm I presume that will give us nothing surprising, but maybe one is unmapped. You should have one or more mimeapps.list files as described here: https://wiki.archlinux.org/index.php/Default_applications#XDG_standard If you grep or just manually skim those for 'notepad', do you find anything? Are they for 'application/octet-stream' or any other 'default' mimetypes? Or can you find wine's internet explorer, whatever its .desktop file would be called?

>>6677 Different anon with same problem here. Don't have mime(1) on my system and couldn't find the package for it, but file -i gives the expected values for webm with binary charset. Running hydrus from console gives an undefined symbol gvfs_is_ipv6 for libgvfsdbus.so, which I have as version 1.32.1+5+gf0d758df-1 in the expected location. After that there's an error on failing to load that module and then there's the console output for the incorrect program. Not sure if it's noteworthy or not, but poking around in the file system shows half of the imported files as being zlib compressed data with no .watever extension. For mimeapps.list, I only have a user override file and the incorrect program is associated with text/plain. Interestingly there wasn't an explicit webm association, but adding one didn't change anything. Thanks for the work so far, it's a pretty comfy program you have here.

Anyone else having issues with GIF's speeding through all their frames in 1-2 seconds? They then remain on their last frame while the seekbar at the bottom works through the time they're actually supposed to take to view these. i.e. a 10 second GIF will speed through all its frames in the first 2 seconds and then remain frozen on the last frame for the remaining 8 seconds. This happened to most of my GIF's, old and new after an update some weeks ago.

>>6724 Thank you for this information. I will add the per-mime launch path options next week to fix this completely. Please let me know then how it works for you. The zlibbed extensionless files are the cached repository update files your client has downloaded. They are stored and managed in the same file system as everything else but obviously aren't user facing. They are all json if you are interested in looking at them closer. >>6725 Can you post or point me at some of these broken gifs, so I can check them my end?

>>6728 Looks like it's not Gif's but is instead webm's. First one doesn't do the thing I was talking about, but it does play faster in client than it does via external player. The rest do what I was talking about

>>6728 >>6730 And again, these did work correctly previously and I'm not on Linux or OSX

>>6731 >>6730 Thank you. I get the problem on the first one (it renders a little fast and then hangs on the last frame for the last ~8s of the video), but the short ones play ok for me, with smooth framerates and correct loops. On first import, in order, my client reckons they have duration/framecounts of: 18.1s, 1086 frames 2.1s, 8 frames 2.0s, 50 frames 2.6s, 52 frames 2.6s, 52 frames With advanced mode on under the help' menu, right-clicking on video will give you a new 'advanced' choice to reparse to possibly correct frame counts and video duration. After running this, the first one goes down to 700 frames (everything else stays the same) and then it renders and loops correctly. Please let me know what duration and frame counts your client thinks it has for those files atm and then run an advanced frame count reparse and see if they change at all. You'll have to reload the images (select and go show selection in a new page to do this quickly) to see the updated values. Also, under help->about, my client's ffmpeg is version N-87043-gf0f4888. I think I last updated it for the Windows release a couple of weeks ago. Does yours say the same? Most older clients have a bunch of videos with incorrect metadata (they were imported with older parsing code that couldn't deal with unusual situations). At some point I will be writing a structure that will automatically reparse all old files with the better newer code, but I still have to do some work getting the current code working right for 99.9% of files. That the cross-section webm gave 1086 guess vs 700 reality even on current code suggests I still have more to do.

>>6734 That did the trick. Old 1,086 frames; 18.1 seconds 43 frames; 2.1 seconds 197 frames; 2.0 seconds 256 frames; 2.6 seconds 256 frames; 2.6 seconds New 700 frames; 18.1 seconds 8 frames; 2.1 seconds 50 frames; 2.0 seconds 52 frames; 2.6 seconds 52 frames; 2.6 seconds ffmpeg version matches as well. Thanks for your help

>>6734 On another note, is there any way to make hydrus recover more gracefully from decompression errors like these? 2017/09/13 14:32:21: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137578293 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:32:30: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137457593 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:33:21: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (136245474 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:35:06: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137959668 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:36:41: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (136247176 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:36:48: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137620860 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:36:55: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (136816125 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:38:08: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (137647785 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:41:10: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (275015787 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:43:14: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (276887313 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack.
2017/09/13 14:50:49: C:\HYDRUS~1\PIL\Image.py:2438: DecompressionBombWarning: Image size (226451652 pixels) exceeds limit of 89478485 pixels, could be decompression bomb DOS attack. It's not my first time with a decompression bomb error but before that only happened while trying to download a file. These managed to succeed and only cause the error when attempting to open them in the media viewer. Result is basically that the computer locks up due to Hydrus using large amounts of memory that increase rapidly; it jumped from ~800MB's to ~1.8GB's roughly 10 seconds into trying to view the file and then jumped to 3.8GB's after another 20-30 seconds or so. Hydrus stays locked up and while you can get taskmanager to pop up (I have a desktop shortcut for it) & try to kill the task, by that point it's pretty much too late since it would take so long to go through. This batch of recent Ke-Ta uploads are the culprits: https://gelbooru.com/index.php?page=post&s=view&id=3872394 https://gelbooru.com/index.php?page=post&s=view&id=3872396 https://gelbooru.com/index.php?page=post&s=view&id=3872424 https://gelbooru.com/index.php?page=post&s=view&id=3872426 https://gelbooru.com/index.php?page=post&s=view&id=3872460 https://gelbooru.com/index.php?page=post&s=view&id=3872465 https://gelbooru.com/index.php?page=post&s=view&id=3872468 https://gelbooru.com/index.php?page=post&s=view&id=3872469 https://gelbooru.com/index.php?page=post&s=view&id=3872483 https://gelbooru.com/index.php?page=post&s=view&id=3872498 https://gelbooru.com/index.php?page=post&s=view&id=3872543 https://gelbooru.com/index.php?page=post&s=view&id=3872545 https://gelbooru.com/index.php?page=post&s=view&id=3872547 https://gelbooru.com/index.php?page=post&s=view&id=3872549 https://gelbooru.com/index.php?page=post&s=view&id=3872550 https://gelbooru.com/index.php?page=post&s=view&id=3872554 https://gelbooru.com/index.php?page=post&s=view&id=3872556 https://gelbooru.com/index.php?page=post&s=view&id=3872557 https://gelbooru.com/index.php?page=post&s=view&id=3872558 https://gelbooru.com/index.php?page=post&s=view&id=3872634 The largest ones are 13859x9927 pixels

switched to running from source in a virtualenv on win7 and got an error on start up Traceback (most recent call last): File "client.pyw", line 20, in <module> from include import ClientController File "C:\Coding\python\hydrus\include\ClientController.py", line 21, in <module> import ClientGUI File "C:\Coding\python\hydrus\include\ClientGUI.py", line 11, in <module> import ClientGUIManagement File "C:\Coding\python\hydrus\include\ClientGUIManagement.py", line 14, in <module> import ClientGUICanvas File "C:\Coding\python\hydrus\include\ClientGUICanvas.py", line 15, in <module> import ClientGUIShortcuts File "C:\Coding\python\hydrus\include\ClientGUIShortcuts.py", line 8, in <module> import wx.lib.flashwin File "C:\Python27\Lib\site-packages\wx-3.0-msw\wx\lib\flashwin.py", line 20, in <module> cc.GetModule( ('{D27CDB6B-AE6D-11CF-96B8-444553540000}', 1, 0) ) File "C:\Coding\python\hydrus\hydrus_env\lib\site-packages\comtypes\client\_generate.py", line 101, in GetModule tlib = comtypes.typeinfo.LoadRegTypeLib(comtypes.GUID(tlib[0]), *tlib[1:]) File "C:\Coding\python\hydrus\hydrus_env\lib\site-packages\comtypes\typeinfo.py", line 478, in LoadRegTypeLib _oleaut32.LoadRegTypeLib(byref(GUID(guid)), wMajorVerNum, wMinorVerNum, lcid, byref(tlib)) File "_ctypes/callproc.c", line 950, in GetResult WindowsError: [Error -2147319779] Library not registered figured out it was because i dont have flash installed. curiously it wasnt a problem when i was running one of dev's compiled releases ended up writing this to get going again diff –git a/include/ClientGUICanvas.py b/include/ClientGUICanvas.py index 9c369aa..781a158 100755 — a/include/ClientGUICanvas.py +++ b/include/ClientGUICanvas.py @@ -30,7 +30,8 @@ import wx if HC.PLATFORM_WINDOWS: - import wx.lib.flashwin + if HC.FLASH: + import wx.lib.flashwin ID_TIMER_SLIDESHOW = wx.NewId() ID_TIMER_CURSOR_HIDE = wx.NewId() diff –git a/include/ClientGUIShortcuts.py b/include/ClientGUIShortcuts.py index 34861b0..8003f44 100644 — a/include/ClientGUIShortcuts.py +++ b/include/ClientGUIShortcuts.py @@ -5,7 +5,8 @@ import wx

[Expand Post]

if HC.PLATFORM_WINDOWS: - import wx.lib.flashwin + if HC.FLASH: + import wx.lib.flashwin def IShouldCatchCharHook( evt_handler ): @@ -13,9 +14,10 @@ def IShouldCatchCharHook( evt_handler ): window = wx.FindWindowAtPointer() - if window is not None and isinstance( window, wx.lib.flashwin.FlashWindow ): + if HC.FLASH: + if window is not None and isinstance( window, wx.lib.flashwin.FlashWindow ): - return False + return False diff –git a/include/HydrusConstants.py b/include/HydrusConstants.py index de276e9..1b50ba7 100755 — a/include/HydrusConstants.py +++ b/include/HydrusConstants.py @@ -667,3 +667,13 @@ sqlite3.register_adapter( bool, int ) sqlite3.register_converter( 'BLOB_BYTES', str ) sqlite3.register_converter( 'INTEGER_BOOLEAN', integer_boolean_to_bool ) sqlite3.register_converter( 'TEXT_YAML', yaml.safe_load ) + + +FLASH = True + +# running from source on windows without flash installed causes an error in comtypes library that isnt handled so catch and disable flash code +if RUNNING_FROM_SOURCE and PLATFORM_WINDOWS: + try: + import wx.lib.flashwin + except WindowsError: + FLASH = False fuck i hope this comes out formatted properly.

>>6741 of course this formatted like shit… i wish i knew how to dump text into a preformatted box

>>6742 code tags "code" "/code" with brackets instead of quotes

Tag parents and siblings do not get along well. An example is character:shinon (sao-alo) being changed to character:sinon (sao-alo). When character:shinon (sao-alo) is entered and shows it will be displayed as sinon (sao-alo), it doesn't add the tag parents from shinon (sao-alo) or sinon (sao-alo). They are both tag parents, but it doesn't input the other tags associated with them if it is displaying one of the tags as another tag.

>>5115 On Linux, when opening Services>Review Services, the window that appears *really* wants to take up the whole screen. It will increase in size until it covers the whole screen, but not beyond.

Saving the session while searching for duplicates crashes the client.

Not sure if this is intended or not, but only swf files play audio in hydrus. I tried with both mp4 and webm files, but neither works.

>>6765 it's intentional

>>6767 Oh. Any way to enable the audio again?

>>6737 I am not sure. This stuff mostly happens at a lower level than I control, in the image libraries I use. Even if I can reliably detect when it will happen, there isn't much I can do about it–an image that decompresses badly needs a bunch of CPU to render. Nonetheless, thank you for the links. I will experiment and try to catch these warnings and add an option to not import bombs. I could also add a 'max resolution' option to the file import options, although that is a slightly different issue. An image that big should only be ~400MB in memory, but the jpeg compression that brings its filesize down to ~10MB or so must have an unusual artifact that is causing the bomb. This may be another case of: >nips being in charge of file formats but I will see what I can figure out. >>6741 Thank you for this report. I will write a similar fix for next week's release. I am guessing my built release bundles some of the Flash ActiveX stuff, or that python doesn't touch the imported 'dll' module in the same way as it would in the source. Check here for 8ch formatting options: https://8ch.net/faq.html#how-do-i-format-my-text The question mark button up top gets you there real quick–I often compulsively recheck that page whenever I mean to do anything clever. >>6751 Thank you for this report. Parent and Sibling logic, particularly when working together, is a real mess. I have a plan to completely overhaul the entire system and add custom ui to make it much easier to edit relationships, but it won't be a small job. Please hang in there for now and add relationships that would make sense in a better system, and I will retroactively fix the missing parents and so on when I catch the system up. >>6757 Thank you for this report. Review services has a mish-mash of old and new ui code that presents differering size calculations. i think it prioritises some of the text widths unnecessarily. If you have a particularly tight screen or are experiencing particcularly unusual layout behaviour, you might like to try forcing its size under file->options->gui->frame locations. Set it to not remember size and then tighten it up. I have a long-term job to update all the gui code to the new sizing system, but it is taking time. >>6763 Thank you for this report. I will be moving duplicate searching to the new 'modal' popups in the near future, to stop gui-interaction while they occur. Those dupe search jobs were always a bit hacky–I recommend you not do anything while they are running, or just put them to run in idle time (check the cog icon on the dupe page). >>6765 >>6767 >>6768 The native video renderer in hydrus is my custom code (in python, wew), which is bodged together with prayers and duct tape. I have not yet written an audio component for it. This will be on the 'big thing to work on next' poll that I will put up after the current downloader overhaul is done, so if you would like this, please vote for it then. At that point, I will add internal metadata for audio as well, so hydrus will display 'has audio' in appropriate places and let you search by it. For now, to hear audio, please double-click running videos to launch them in your OS-default video player.

These three images seem to have rotated counter clockwise, but the aspect ratio stayed the same. The thumbnail shows show them as they should be. Exporting them out also has them rotated, but with the correct aspect ratio.

>>6786 >counter clockwise Clockwise, I mean

>>6787 >>6786 Thank you for these examples. Support for EXIF-rotated images is not complete yet. Part of the rendering engine does it, but the file parsing doesn't, so they disagree. I hope to have complete support for different cases in the coming weeks, and then I will write a maintenance routine to re-parse existing files for rotation (and hence update their caches resolutions, which is part of the problem you are seeing) retroactively. Tell me if I am being stupid, but hasn't 8chan/my browser made a similar mistake here? The thumbs look correct, but they load +90, -90, +90 degrees when I expand them. The stated 3000x1740 res is also incorrect in the same way, wew.

>>6824 The thumbs here actually aren't correct, the expanded images are(see the text). Opening in a new tab doesn't have the correct orientation again. Hydrus thumbnails are correctly oriented though. Also this wasn't an issue in Hydrus when I imported them a month and a half ago, it's more recent.

>>6786 >>6824 >>6825 I don't have this problem but the source is twitter https://twitter.com/arapaima55giga/status/893426886235246594 so trying importing the original files and see if that works. I took the copies on gelbooru and while they don't match the twitter versions (gelbooru's come from danbooru), I don't have the rotation problem either.

When I select gifs, the client crashes with a segmentation fault before it can render the first frame in the preview box I can't open them in any other way, as selecting them shows the preview and hydrus crashes as soon as the first frame is loaded This actually happened in 274, but when I finally managed to upgrade to that version and discover the issue, 275 was released already The crash still persists though

I've installed hydrus on win10 and it simply refuses to run. I can open the the server console but attempting to start the client does nothing. It appears in the task manager for a moment, then disappears, without any feedback. I did run it as an administrator. What could be at fault here?

>>6845 >>6825 Thanks. I get different rotations depending on whether I 'expand' the images inline vs opening them in a new tab, wew! I will try to get all the pipelines doing what I think is correct and then we'll reparse this stuff and iron out lingering mistakes at that point. Thank you for the examples, as they will make testing on my end that much easier. >>6853 Thank you for this report. I am sorry about your problems. Please flip file->options->media->disable OpenCV for gifs to whatever it currently isn't and let me know if that changes anything. What happens if you open a webm or mp4? And what version of PIL, Pillow and OpenCV do you have listed under help->about? And which OS are you on? >>6854 Hey, I am sorry you are having problems. Please check your install_dir/db folder for a crash.log file. If it exists, what does it report inside? If no such file exists, please go back to install_dir, and then shift+right-click on some whitespace->open command window here and then type 'client' and hit enter on the terminal window that will pop up. If it gives an error traceback there, please take a screenshot and post it here or send it to my mail or whatever. Some users with similar symptoms have also been hit by over-zealous anti-virus programs. If you run third-party anti-virus, please check that it hasn't quarantined part of the hydrus zip or installer or your install directory. Also, is hydrus installed to a protected location like C:\Program Files? It needs to write to its own directory on a clean first boot. Running as admin might get around this, although it is obviously not ideal.

>>6858 There is no crash log file, this is what the terminal turned out.

>>6862 Time to get glasses.

>>6858 >Thank you for this report. I am sorry about your problems. Please flip file->options->media->disable OpenCV for gifs to whatever it currently isn't and let me know if that changes anything. Disabling OpenCV for gifs fixed it >What happens if you open a webm or mp4? Everything works fine, with and without OpenCV for gifs >And what version of PIL, Pillow and OpenCV do you have listed under help->about? OpenCV: 3.2.0 PIL: 1.1.7 (though I have the option to load images with it disabled) Pillow: 3.4.2 >And which OS are you on? I'm the same person of >>6631 >>6632 >>6641 OpenCV keeps being an issue

>>6863 >>6862 Thank you for the update. I'm not familiar with powershell, but I guess you have to .\ to execute, wew! Apparently this is a new change, and if you would like to roll back to cmd, you can set it under Windows' settings->personalization->taskbar. Anyway, please download this debug build and do the same thing: https://www.mediafire.com/file/v1ugfjwyt3ulzfv/Hydrus%20Network%20276%20debug%20-%20Windows%20-%20Extract%20only.zip It will explicitly write a lot of stuff to the terminal about how the boot is going, and should give us more info. >>6866 Thank you for the follow-up. I am not sure what is doing this, but OpenCV has been a common pain, despite how good it is otherwise. Some Windows users had problems with OpenCV a while ago due to a bad set of Nvidia drivers–OpenCV uses OpenCL to do some rendering hardware acceleration, and a bug in Nvidia's OpenCL code was causing blue screen! It might just be possible that you have a similar problem? If your GPU driver allows it, maybe try turning off OpenCL/CUDA support for client.exe. This may permit OpenCV to work again.

>>6899 I'm using a 64-bit system, if that is relevant in any way.

>>6899 >It might just be possible that you have a similar problem? If your GPU driver allows it, maybe try turning off OpenCL/CUDA support for client.exe. This may permit OpenCV to work again. I have AMD hardware inside my laptop, and the "GPU" is the integrated card (it's a relatively old machine) I can't really turn off support for anything, especially since I run hydrus from source (i.e. client.py) OpenCV used to work fine, at least on version 2 I'll see if I can downgrade it and give results

>>6905 Hydrus 276, OpenCV 2.4.13, enabled for gif Everything works fine, the client starts and selecting gifs shows them with no issues

Anyone else having this issue? I can't see the checkmarks… I thought the removal of the buttons was annoying but at least they still had text… with this I can't do anything…

>>6911 >no picture

>>6903 Thank you. I believe we had a similar problem a while ago with a different user who had a unicode character in his PATH environment variable, which the version of python I currently use cannot deal with. I think it was something including his username. If you are not familiar with your PATH, just google it a bit. It is basically a list of places windows checks when it runs things. You can see how to find it here: https://www.computerhope.com/issues/ch000549.htm If you are not very familiar with what it is, be very careful. It is a critical part of Windows and should not be edited randomly. Please just check it for now–are there any unusual characters in your PATH? If not your PATH, are there any other system level paths on your machine that might have a unicode character? Your username or anything else? >>6905 >>6906 Great. I am not sure what happened here then. Maybe the new OpenCV hooks were interacting badly with your graphics stuff. Please let me know if you discover anything new. >>6911 >>6912 I do not know what is going on here, and I don't think I have seen it before either. I don't think I changed anything on my end (with wxPython) that determines how these things are shown. Which OS are you on? Do you have any window managers active that alter gui elements? Has hydrus ever displayed correctly for you, or has it always had some corruption of some sort? If you open up the options menu, do the checkboxes and other elements on that also display badly, or is it just limited to the pages' left-hand management panels?

A new problem, how exciting! It started somewhere in one of the 27x updates, I don't know which one though… before that it displayed correctly. My OS is Mint 18. I don't know if I have window managers but I don't think so… The options screen looks like pic related. Note that the currently selected tab is invisible in the list of tabs.

>>6936 meant to reply to >>6930

When using "View This file's duplicates" to look at a file's duplicates, the resulting screen isn't ordered with the Sort by system. In this picture, the largest file sits in the middle, despite this page ostensibly being sorted by filesize. Also, having a regular page for the "View this file's dupes" page seems a little off. Supposing you previously ran the file dupes on pictures A and B, before opening up file C and viewing it's dupes, it's extremely easy to overwrite the relationship AB when creating a relationship AC and BC without being aware of it. This becomes an issue with tens of files to a window. Ideally, the file C should be kept apart from the rest by the UI and made clear its relationships between itself and other files and their relationships, but that's not really a bug so sage for backseat programming.

Anyone's tumblrshit stop working?

>>6967 Tumblr changed their URLs for raw images, they're now on data.tumblr.com instead of media.tumblr.com

>>6968 with https errors to boot

>>6968 As of…today? I had shit working yesterday

>>6973 Sometime within the last week >>6969 Apparently s3.amazonaws.com/data.tumblr.com works correctly

>>6975 >>6973 two days ago

Updated from 268 to 277 because Sankaku broke, but now few of the UI elements render properly. Most are usable, but checkboxes in the booru import page (if not other places too) are non-functional. System: Linux Mint 18.2 (base - Ubuntu 16.04) Cinammon Edition, Skylake i5 processor, AMD GPU (proprietary drivers).

>>6980 Hey I have the same problem, here >>6912 and >>6936 Also Mint…

>>6980 >>6936 Can you please try extracting a fresh empty v268 and then comparing the library version numbers in help->about to your v277? I am guessing that wx or something got updated, perhaps in my recent Linux dev environment rebuild. If so, that at least gives us some information about this. Then, please open a terminal and go: python
import wx
wx.__version__
exit() It may be the wx import statement fails, but it would be useful to know if Mint has a system wx as well that is perhaps conflicting here. >>6963 Thank you, I will see if I can fix this for next week. I agree about the dupe stuff. This first version I made has a lot of limitations, and most of my time went into the db-end of doing discovering dupes. The actual ui to make decisions about dupes is a lot less sophisticated, and I re-used existing ui where I could to save time. I hope a subsequent iteration of the system will have more specialised ui. I'd love to write some drag-and-drop/draw-arrows-between ui controls in future to help this sort of stuff, rather than always dealing with grids and lists. (drawing linked networks dynamically would be great for tag siblings and parents as well!) >>6967 >>6968 >>6969 >>6973 >>6975 >>6979 Thank you for this report. Yeah, as 9c435e says, they rearranged their CDN and have a mickey-mouse ssl cert going on as well. I have this fixed for v278, please let me know if it gives you any more trouble.

Hydrus 277: This client is the media management application of the hydrus software suite. FFMPEG: 2.8.11-0ubuntu0.16.04.1 OpenCV: 3.3.0 openssl: OpenSSL 1.0.2g 1 Mar 2016 PIL: 1.1.7 Pillow: 4.2.1 python: 2.7.13 sqlite: 3.16.2 wx: 3.0.2.0 gtk2 (classic) Hydrus 268: This client is the media management application of the hydrus software suite. FFMPEG: 2.8.11-0ubuntu0.16.04.1 OpenCV: 3.2.0 openssl: OpenSSL 1.0.2g 1 Mar 2016 PIL: 1.1.7 Pillow: 4.0.0 python: 2.7.13 sqlite: 3.16.2 wx: 3.0.2.0 gtk2 (classic) Python wx test: cfalcon@House-of-Hype ~ $ python Python 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import wx >>> wx.version '3.0.2.0' >>> exit() cfalcon@House-of-Hype ~ $ On a probably unrelated note, my version of OpenSSL is not the version listed in Hydrus. My version is 1.1.0f, the latest version on OpenSSL.org. This is probably just because I download the tarball on Github. I think 1.0.2g is still installed somewhere on my computer, but the if multiple exist then the default is the new version. On a possibly related note, other programs (more specifically, a few source-installed emulators) have given me error messages that tell me that the version of wx the program was compiled with is not the same as my library's current version. Reinstalling the offending libraries with apt-get and recompiling the program did not stop the messages, but the programs all appear to run as expected anyways.

>>6991 >>6980 Also, right click->open externally now tries to open pngs, jpegs, and animated gifs in Wine's Internet Explorer instead of the default xviewer (which looks like a recent fork of eog (eye of gnome), the default image viewer in Mint 17.x). I get a wine error (something about no Windows program configured to open this file type) when I try to right click-> open externally for webms. On 268 and earlier, I usually opened webms externally so that the sound would work.

>>6987 hydrus client 268, using network version 18: FFMPEG: 2.8.11-0ubuntu0.16.04.1 OpenCV: 3.2.0 openssl: OpenSSL 1.0.2g 1 Mar 2016 PIL: 1.1.7 Pillow: 4.0.0 python: 2.7.13 sqlite: 3.16.2 wx: 3.0.2.0 gtk2 (classic) hydrus client 276, using network version 18: FFMPEG: 3.2-static OpenCV: 3.3.0 openssl: OpenSSL 1.0.2g 1 Mar 2016 PIL: 1.1.7 Pillow: 4.2.1 python: 2.7.13 sqlite: 3.16.2 wx: 3.0.2.0 gtk2 (classic) Python: ~ $ python Python 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import wx >>> wx.version '3.0.2.0' >>> exit() ~ $ >>6992 >Also, right click->open externally now tries to open pngs, jpegs, and animated gifs in Wine's Internet Explorer instead of the default xviewer I used to have this too for both hydrus images and other images a while ago, so I uninstalled Wine. I just now noticed that I can't open externally from v276 at all.

>>6987 >>6993 I just redownloaded them all and the problems appear in version 271.

Regarding the thread watchers auto naming: Regardless of the settings, opening an old session with lots of thread watchers will cause them to lose my custom names for them. I would love it for this to be fixed, as I have a lot of sessions with old thread watchers, many of which the names are not obvious.

>>6993 >>6991 >>6995 Thank you for this. Looks like wx isn't it, but I will check the v271 changes to see what I did. >>6992 >>6993 Please check the new overrides for 'open externally' under options->files and trash. Seems like some flavours of Linux are all over for this stuff, so you can now set it yourself. Please let me know if you run into any more trouble with this. >>6996 I will see if I can do this simply, thank you for the report.

>>6987 >Thank you for this report. Yeah, as 9c435e says, they rearranged their CDN and have a mickey-mouse ssl cert going on as well. I have this fixed for v278, please let me know if it gives you any more trouble. Will this essentially reset URLs? Just curious

>>6997 The file open for all programs was set to `default:xdg-open "%path%"` (if I messed up the formatting, ignore the backticks). I opened a jpeg with that command from a terminal, and it loaded in xviewer as expected. So I tried setting the open externally command for image/jpg to `xviewer "%path%"`. Open externally ceased to function. So I tried changing the double quotes to single quotes. Xviewer still didn't open, but when I changed the program from xviewer back to xdg-open, xdg-open opened my jpg in Wine's IE, just as it did when %path% was in single quotes. I noticed another new thing last night: file-> import files no longer correctly displays my system's graphics. It shows something, but it's not Wine's graphics and it's not Mint's usual graphics. IIRC it's something used by GNOME (which Cinnamon is a fork of), but the last time I used a non-Mint distro was about 6 - 8years ago. I normally wouldn't care about this, but I figured it might be useful debugging information. I'll upload some console error logs in a little while. Thanks for working on this program, thanks for sharing it with the rest of the Internet, and a huge thanks for supporting Linux users and all of their weird setups.

>>7000 >just as it did when %path% was in double quotes correction

>>7000 Apparently I can't post the error logs. I'm not allowed to attach .txt files to my post and if I post the error logs here, 8chan experiences a buffer overrun and doesn't post my reply. Speaking as someone who doesn't know Python, but does know a few other languages, there seems to be some interesting information contained in the 846kB console output log (I had to copy-paste, since the >> operator didn't catch most of the actual errors). Possibly even more interestingly, 268 gives errors too, but they seem to be different errors and they only show up on program startup.

>>7002 not dev ofc, but maybe you could upload the logs here: https://pastebin.com you can set pastes to private so that they are only find-able with the link

>>6993 >>6991 Ok, I had a look at the commit here: https://github.com/hydrusnetwork/hydrus/commit/e7449281e6f66400ac3ada9658ec48d7cb46f30c There is nothing too special in the code, but the actual Linux release went down from 140MB to 102MB. Checking my release post, this is indeed the week when I had to rebuild my Linux machine. Afaik, I now use a clean Ubuntu 17.10, whereas the old system, while I think it was 17.10, was updated from 15.something originally. I presume that all the library updates I got with a fresh build of my dev environment is now clashing with your Mint (16.04). It could also be a new version of PyInstaller packnig things differently, as I see a different format for the cv2 .so stuff as well in the two releases. In any case, I am guessing some GTK-something isn't presenting borders to your older window system properly. I assume in your tests you have been making fresh installs. I was going to suggest trying a 'clean' install, in case the spare .so files from the v270 were confilcting with the v271, but if a fresh extract doesn't work either, that can't be true. I do not know much about Linux, and I know less about Mint. Are they bringing out a new version based on the new Ubuntu? Do my thoughts on this likely being an old and new Ubuntu conflict sound reasonable? I believe I know a Mint user who runs hydrus from source. If you are comfortable running pip a bunch of times, you could try this yourself as well. You already have wx, and that is usually the biggest pain. Here is the guide: http://hydrusnetwork.github.io/hydrus/help/running_from_source.html

>>7008 Hey, >>6995 here. I have the same 'default:xdg-open "%path"' and stuff as >>7000, I didn't try all those things they did. I don't know where I can find an error log, but launching from terminal with ./client gives me: >(client:8643): Gdk-CRITICAL **: IA__gdk_window_get_origin: assertion 'GDK_IS_WINDOW (window)' failed a couple of times and then things like: >(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_pixels_with_length: assertion 'GDK_IS_PIXBUF (pixbuf)' failed >(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_rowstride: assertion 'GDK_IS_PIXBUF (pixbuf)' failed >(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_n_channels: assertion 'GDK_IS_PIXBUF (pixbuf)' failed >(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_width: assertion 'GDK_IS_PIXBUF (pixbuf)' failed >(client:8583): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_height: assertion 'GDK_IS_PIXBUF (pixbuf)' failed literally hundreds of times, occasionally interrupted by the message: > (client:8583): WARNING : Invalid borders specified for theme pixmap: > /usr/share/themes/Mint-X/gtk-2.0/images/notebook/tab-top-active.svg, >borders don't fit within the image or: > (client:8583): WARNING : Pixbuf theme: Cannot load pixmap file /usr/share/themes/Mint-X/gtk-2.0/images/notebook/notebook.svg: Couldn't recognize the image file format for file '/usr/share/themes/Mint-X/gtk-2.0/images/notebook/notebook.svg' In response to your post: - Yes I have been making fresh installs. Well, "install", I just download and extract. - I don't know much about Mint either. Afaik it is based on Ubuntu. The thing with Mint is that they are always very slow to update anything. So your thoughts may be correct. But I don't know anything about computer stuff so there's that. - I will try running from source and report back. - Thanks for all your attention to this problem.

>>7010 >spoilers There were double asterisks there, guess that's how 8chan does spoilers…

>>7008 >>7010 I tried running from source and it worked. Though I got this error: >libpng warning: iCCP: known incorrect sRGB profile and this one >(client.pyw:10820): Gdk-CRITICAL **: IA__gdk_window_get_origin: assertion 'GDK_IS_WINDOW (window)' failed during startup, everything seemed fine, though a bit laggy. In the second launch the lag had disappeared, I have that sometimes with my pc. Two comments about your guide: 1. I had to apt-get install python-dev to be able to pip install lz4 and psutil (and it might be needed for other packages too, Idk). 2. 'pip install package' will not upgrade an existing installation, you need to use 'pip install package –upgrade' for that. My normal installation still doesn't work though.

>>7012 >– upgrade this should be - and another - and then upgrade

>>7012 'open externally' works too on this install from source btw (the media viewer still doesn't work though but that's unrelated to this)

>>6996 >>6997 The new option for not showing the 404 and dead marker doesn't seem to improve this. When the old session is loaded, all the tabs have their old names, but as they initialize, they change name to "[dead] Thread Watcher"

>>7010 >>7012 >>7014 Hey, it's the other Mint user here. My error messages when running the tarball were slightly different, but the program seems to run correctly (correct icons, checkboxes work, open externally works) from source. Running from source is a bitch on Mint. In addition to the BS that >>7012 did, I had to have pip upgrade lz4 after installing it or else it would fail to import lz4.block because _reasons_. I also had to sudo -H pip install a few of those modules. I didn't even know sudo _had_ flags, I always thought that the only arguments it takes are other commands. Just an FYI, Mint 18 is pretty damn close to Ubuntu 16.04. If we're having trouble, they're probably having trouble. 18 seems to diverge a bit more from its Ubuntu base than earlier Mints did, but it's probably still close enough. How exactly do you build the Linux version? I don't see a makefile or anything. Is it something I can do and share with my Mintian brothers?

>>7030 >How exactly do you build the Linux version? I don't see a makefile or anything. Is it something I can do and share with my Mintian brothers? What do you mean? Have you downloaded and unpacked the source .zip or .tar.gz? Open terminal and cd to the directory that it is opened in, then just do python client.pyw, that's all there is to is (at least that worked for me). If you want to be able to open it from anywhere just make an alias.

>>7032 >the directory that it is opened in I meant the directory that you get when you open the compressed thingy sorry.

>>7032 I haven't tested yet, as my hydrus folder is backing up, but I would assume that copying the source install over the old hydrus install wouldn't work right. I would guess that all of the so files in the regular hydrus folder would mask your system's shared objects. Yes, it can be fixed with one terminal command, but there are a bunch of other subdirectories which might give similar issues. Also, when running from source the client would occasionally hang up for a few seconds. On the regular Linux install, I rarely see that except when I've been running hydrus for several days straight, trying to download all files from a huge tag on Gelbooru. Maybe it's just coincidence, but even if it's not, if I can make it easier for people to use Linux by taking literal seconds out of my week to run a single script or two, I don't see why I shouldn't do it if it's that fast.

>>7012 Thanks. These sRGB ash GDK_BLAH errors are what I normally see when I run from source. There are a few of these lingering issues on each platform. I am not sure which of them are my fault and which are down to the libraries, but I'm generally deprioritising them until they cause a real problem. If your source works good, I encourage you to keep using it, and maybe try the built release again when Mint next updates. The other Mint guy I knows just gets the source straight from git and is able to update every week in like three seconds. Thank you for the notes on the extra apt-get and pip stuff. >>7027 I am afraid I do not have a fix for this yet. The custom names will be overwritten for now. >>7030 >>7032 >>7033 >>7034 Yeah, this shit is an unending headache for me as well. I can appreciate there are many different players involved who all have different interests, but sometimes I wish pip and the other stuff had some 'just make it work, would you?' options, or maybe even, shock horror, a gui. I build using PyInstaller. This basically wraps the code up into an exe with a python environment and all the required dlls. It is magic on a level far deeper than I understand, but it is generally pretty good at its job. Given the various reports I have now had, I presume the virtual environment it is running in is now the cause of the problem xdg calls in some people's Linuxes. I have a build script I run every week that does some environment specific stuff like copying the statis folders into the build and wrapping it all up into a correctly named archive and so on. The magic line to run is: pyinstaller --windowed --hidden-import appdirs --hidden-import six --hidden-import packaging --hidden-import packaging.version --hidden-import packaging.specifiers --hidden-import packaging.requirements --name="client" hydrus/client.py Your hidden import stuff may be different–that's just what I have found I need. I cannot say with any great authority (it works on Windows, at least), but I think you can run from source in a normal install without any conflicts. I include the .py files (and the include folder) in the normal releases for this exact reason–you can either run the exe or the script as you like. Python on its own won't look for the .so files like the exe would–it'll look for a .pyd or something in your local python's site-packages or whatever, and that'll eventually go back to your system .so stuff as per normal. I am interested if you discover otherwise though! I do like this stuff, even if it isn't my natural environment and can be a trudge at times.

>>7041 >I build using PyInstaller. This basically (…) per normal. I must admit I don't understand anything about what it means to "build" it… I just executed the client.pyw script from the source zip with python… is that a bad idea? I tried doing the same in the hydrus network folder you get from a normal install, and I got this error:
~ $ python client.pyw
Traceback (most recent call last):
File "client.pyw", line 13, in <module>
from include import HydrusData
File "~/Pictures/hydrus network/include/HydrusData.py", line 8, in <module>
import HydrusSerialisable
File "~/Pictures/hydrus network/include/HydrusSerialisable.py", line 2, in <module>
import lz4.block
ImportError: No module named block
My lz4 version is 0.10.1. If I try to import lz4 in a normal python shell, I can just do things like "from lz4 import block" or "import lz4.block" and I don't get error messages. So this is a bit odd.

>>7041 When you have the time to look into it those libpng warnings are due to the fact that the static files for the icons of flash, pdf and zip media have ICC Profiles that libpng (which I assume is a dependency of either OpenCV or Pillow) doesn't like. Best bet if you'd like to avoid the warning would be just to strip the profile from those PNGs. Of course this doesn't solve the problem of wild PNGs imported to hydrus throwing the same warning but meh, it doesn't mean much anyway. Here's a bit more detail on it: https://wiki.archlinux.org/index.php/Libpng_errors

>>7041 Pics spoilered due to very NSFW thumbnails All upcoming information comes from 277, since I have both the source and executable archives already downloaded. Copying the source files over my old install doesn't work, but if I delete the old .so files in the main hydrus directory everything I've tested works as expected (this hold true for 278 as well). Pyinstaller build from source mostly works (details below), but only if I copy the contents of the dist/client/ folder over either the source archive or the executable archive. Otherwise, it complains about missing icons and won't start. I just tried a PyInstaller build of 277 (since I have both the source and executable archives already downloaded). It's still weird, but it works a hell of a lot better than the regular Linux executable, though not as well as running straight from source. The default xdg-open from Hydrus still opens in WINE IE, but manually setting image/jpeg (probably the others too) to xviewer will make it work right. Granted, some of xviewer's icons don't render if I do that, but that's probably xviewer's problem and not yours. Most UI elements render as expected, and checkboxes function again. File open icons are still using the weird ones I mentioned earlier. Duplicate processing no longer appears on the right click menu, for some reason. AFAICT, there's no way to set duplicate relationships with the build I just made, though the duplicates search page appears to function as expected. The dupe processing filter may work, but I have no idea how to control it. I got a few errors during the build, posting them below. I'll try to make a Mediafire account or something and post an alternate Linux build in the new version release threads (hence the tripcode). cfalcon@House-of-Hype ~/.hydrus $ pyinstaller --windowed --hidden-import appdirs --hidden-import six --hidden-import packaging --hidden-import packaging.version --hidden-import packaging.specifiers --hidden-import packaging.requirements --name="client" hydrus-277/client.py
81 INFO: PyInstaller: 3.3
81 INFO: Python: 2.7.12
81 INFO: Platform: Linux-4.8.0-53-generic-x86_64-with-LinuxMint-18.2-sonya
83 INFO: wrote /home/cfalcon/.hydrus/client.spec
85 INFO: UPX is not available.
86 INFO: Extending PYTHONPATH with paths

...

14197 INFO: Processing pre-find module path hook site
14197 INFO: site: retargeting to fake-dir '/usr/local/lib/python2.7/dist-packages/PyInstaller/fake-modules'
22114 INFO: Loading module hooks...

...

22150 INFO: Removing import of PyQt4.QtCore from module PIL.ImageQt
22150 INFO: Loading module hook "hook-wx.lib.activex.py"...
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/wx-3.0-gtk2/wx/lib/activex.py", line 35, in <module>
import ctypes.wintypes as wt
File "/usr/lib/python2.7/ctypes/wintypes.py", line 19, in <module>
class VARIANT_BOOL(_SimpleCData):
ValueError: _type_ 'v' not supported
22243 INFO: Loading module hook "hook-lxml.etree.py"...

...

23287 INFO: Loading module hook "hook-setuptools.py"...
23288 WARNING: Hidden import "setuptools.msvc" not found!
23288 INFO: Loading module hook "hook-PIL.SpiderImagePlugin.py"...


Forgot to post images. This is what Hydrus looks like when I build it and copy+paste the output folder over the extracted source and executable folders. Again, very NSFW thumbnails from the test DB I pulled from Gelbooru.

If I download a picture from e.g. danbooru then the source url is not https.

>>7063 Some of the defaults are old and have not been updated to reflect https yet. I am not sure if danbooru has been updated yet, but I will check. In any case, please go services->manage boorus then select danbooru and change the search url to 'https'. Just add the 's', hit ok, and it should thereafter fetch https urls. I just tested this on my own cleant and it seems ok, but let me know if it gives you any trouble. >>7052 Thanks, this is really helpful. I will put some time aside to sort this out. >>7059 >>7060 Yeah–there may be ways to inform pyinstaller about non-python stuf like the 'static' and 'help' folders that you eventually want in the build directory, but I just copy it over in a script in the same way. Make sure you don't ever copy a real database into your empty build db directory–I almost completely fucked myself once doing that. Could the duplicate stuff be that you are not in advanced mode (help->advanced mode)? New clients won't start in advanced. I get similar errors when I build. There are ten kinds of voodoo that pyinstaller has to do to get OS-specific stuff to work, and it logs all the problems it finds. So far, my rule has been that if it boots on the other side, I won't complain. Grats for figuring this out, and thanks again for making your release available.

Just for shits and giggles, I tried manually setting the open externally program to xdg-open. Didn't fix anything, but I didn't expect it to. >>7069 As it turns out, not being in advanced mode is exactly why dupe processing didn't work.

>>7069 >services->manage boorus Thanks that worked !

Not sure if file error or hydrus error, but file related gives me an unsuported mime error.

found a tiny bug in this window, if you remove a tag by double-clicking it, the text below doesn't update if you click the delete button, it does

With version 282, the main GUI (search, sort, tags) is resetting in size on each new tab. I changed no settings. This is also new behavior, and I also realize I have no clue how to edit the GUI elements, the changes I'm making don't effect the GUI options menu.

>>7261 this started working again, must have something to do with running off an external.

Don't know if this is a real bug or i3 beeing stupid but when hydrus starts I get these boxes, one for each tab I have open. They work normally (I can modifiy what filters apply to the different tabs) but its really annoying. When I used hydrus on Windows these boxes would appear but would dissapear once hydrus was fully loaded.

>>7221 Thank you for this example. This is a 'ZWS' flash file, or one that is compressed with an unusual flavour of LZMA. I have tentatively patched the flash parsing library I use to decompress the header and this file now seems to import. It reckons (800x600, 1.7s, 50 frames), which seems reasonable for a game. Please let me know if you still have trouble or if you find any other flash files that will not import. I am not sure I have the LZMA stuff pinned down. >>7259 Thank you for this report. This should be fixed for tomorrow's release. >>7261 >>7266 Some of the main gui size preference code is a bit old. The 'splitter' positions are only remembered from the current page on client exit, if I remember right. This may be what was happening here. Let me know if you discover more here, in any case. >>7276 I have not had much success with Linux and 'floating' windows. It sounds like you've transferred over, but a fresh non-Windows client defaults to 'embed' these dropdowns to avoid the problem. Please check options->gui->always embed a/c dropdown results window and restart the client. The 'hover' windows in the media viewer can also flicker and move offscreen in some Linux window managers. There's a BUGFIX option on the same options page to just have them display all the time if you get this. Let me know how you get on. I've bashed my head against this stuff several times, but if there's some interesting new behaviour or clues on what is going, I'm willing to have another go.

>>7286 >Please check options->gui->always embed a/c dropdown results window and restart the client. That solved it, thanks!

I just upgraded to 284 (MintTeaFresh's Linux build). Hydrus then checked 3 of my booru subscriptions, and declared all of them dead. Two of them haven't found new images in over a year, so that's not surprising, but one grabbed something less than a month ago.

>>7372 Can you please check the respective subscription's 'edit check timings' button? Any subs that updated from the old system got different 'dead' timings based on the old static subscription period. It may be that has 'dead of < 1 file per 14 days' or something.

It seems like related tags isn't working right anymore. Now it just seems it picks a random image it knows and suggests all the tags from that image now.

>>7399 I don't think I changed any of that code. It has never been excellent, particularly if the search distances in options->tags are short. What happens if you open manage tags over and over on the same file–do you get substantially different results every time? What if you click the 'thorough' button? If so, you might like to try bumping up the search time, as your mappings have probably expanded and it might not be searching the whole search space deep enough to get good results. Can you go into any more depth on the sorts of bad results you are getting? Different 'sample' tags on the file work better than others. Also, some big tags (like 'series:touhou') have such a large imprint on the db that they can muddy any other specificity. A future iteration of the system could add weights on input/output tags based on 'usefulness', although I am not sure what that exactly means in statistical terms atm.

Database Traceback (most recent call last):
File "/include/HydrusDB.py", line 523, in _ProcessJob
result = self._Read( action, *args, **kwargs )
File "/include/ClientDB.py", line 7985, in _Read
elif action == 'media_results_from_ids': result = self._GetMediaResults( *args, **kwargs )
File "/include/ClientDB.py", line 5076, in _GetMediaResults
hash_ids_to_hashes = self._GetHashIdsToHashes( hash_ids )
File "/include/ClientDB.py", line 4900, in _GetHashIdsToHashes
( hash, ) = self._c.execute( 'SELECT hash FROM hashes WHERE hash_id = ?;', ( hash_id, ) ).fetchone()
TypeError: 'NoneType' object is not iterable I imported a file without a hash. Things broke.

latest download, on linux, specifically slackware. every few minutes this happens. DBException
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)
Traceback (most recent call last):
File "include/HydrusThreading.py", line 146, in run
self._callable( self._controller )
File "include/ClientDaemons.py", line 46, in DAEMONCheckImportFolders
import_folder = controller.Read( 'serialisable_named', HydrusSerialisable.SERIALISABLE_TYPE_IMPORT_FOLDER, name )
File "include/HydrusController.py", line 370, in Read
return self._Read( action, *args, **kwargs )
File "include/HydrusController.py", line 120, in _Read
result = self.db.Read( action, HC.HIGH_PRIORITY, *args, **kwargs )
File "include/HydrusDB.py", line 846, in Read
return job.GetResult()
File "include/HydrusData.py", line 1774, in GetResult
raise e
DBException: UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)
Database Traceback (most recent call last):
File "include/HydrusDB.py", line 523, in _ProcessJob
result = self._Read( action, *args, **kwargs )
File "include/ClientDB.py", line 8004, in _Read
elif action == 'serialisable_named': result = self._GetJSONDumpNamed( *args, **kwargs )
File "include/ClientDB.py", line 5031, in _GetJSONDumpNamed
return HydrusSerialisable.CreateFromSerialisableTuple( ( dump_type, dump_name, version, serialisable_info ) )
File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
obj.InitialiseFromSerialisableInfo( version, serialisable_info )
File "include/HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo
self._InitialiseFromSerialisableInfo( serialisable_info )
File "include/ClientImporting.py", line 1951, in _InitialiseFromSerialisableInfo
self._path_cache = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_path_cache )
File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
obj.InitialiseFromSerialisableInfo( version, serialisable_info )
File "include/HydrusSerialisable.py", line 153, in InitialiseFromSerialisableInfo
( version, serialisable_info ) = self._UpdateSerialisableInfo( version, serialisable_info )
File "include/ClientImporting.py", line 3334, in _UpdateSerialisableInfo
if os.path.exists( seed_text ):
File "genericpath.py", line 26, in exists
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)


Database Traceback (most recent call last):
File "include/HydrusDB.py", line 523, in _ProcessJob
result = self._Read( action, *args, **kwargs )
File "include/ClientDB.py", line 8004, in _Read
elif action == 'serialisable_named': result = self._GetJSONDumpNamed( *args, **kwargs )
File "include/ClientDB.py", line 5031, in _GetJSONDumpNamed
return HydrusSerialisable.CreateFromSerialisableTuple( ( dump_type, dump_name, version, serialisable_info ) )
File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
obj.InitialiseFromSerialisableInfo( version, serialisable_info )
File "include/HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo
self._InitialiseFromSerialisableInfo( serialisable_info )
File "include/ClientImporting.py", line 1951, in _InitialiseFromSerialisableInfo
self._path_cache = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_path_cache )
File "include/HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple
obj.InitialiseFromSerialisableInfo( version, serialisable_info )
File "include/HydrusSerialisable.py", line 153, in InitialiseFromSerialisableInfo
( version, serialisable_info ) = self._UpdateSerialisableInfo( version, serialisable_info )
File "include/ClientImporting.py", line 3334, in _UpdateSerialisableInfo
if os.path.exists( seed_text ):
File "genericpath.py", line 26, in exists
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 55-56: ordinal not in range(256)


>>7435 also, hydrus seems to crash alot when minimized during these errors.

>>7395 Thanks, it looks like it'll behave again if I play around with those settings.

>>7433 I am sorry you are having this problem. Can you explain more what lead up to this? Was this a normal file import that somehow has broken things, or did you import a file by interacting with the database directly with a SQL program? Every file should have a hash. Have you recently had any power outages or hard crashes of the client during importing? >>7435 >>7438 Thank you for this report. I am sorry for the inconvenience. Please hit services->pause->import folders for now. This will stop the error from spamming you. I will look into the traceback and try to fix the error for next week. Please try unpausing your import folders then and let me know if it works again.

>>7473 error spam continues if I pause and unpause. also, I think I may have mistook >>7438 for hydrus closing because of inactivity. the spam is still annoying though.

>>7473 >>7433 here I'm not 100% sure what happened (only discovered it when a search failed) but I think my computer suspended/went to sleep while hydrus was importing a file causing an entry to be made without a hash included. Checking database integrity found no problems. A quick file integrity check "corrected" the problem but that led to another issue where majority of new imports were failing because the db says it had already deleted that file from the db. In the end it's not a big deal. I've created a fresh db.

Sometimes I turn off my remote file/tag server. If I accidentally try to manage the service, Hydrus becomes unresponsive and I have to wait for the connection to timeout to be able to do anything. It's low priority, since it eventually recovers, but it would be nice to have a way to cancel the connection. Additionally, is the connection attempted on a separate thread? Why does the UI freeze?

Hello dev. When you map a 'tag' on 'namespace:tag' with tag sibling service,you can't find pics with 'namespace:"anything"'. For example when you map 'kiniro mozaic' on 'series:kiniro mozaic', pics appear with 'series:kiniro mozaic' but they don't appear with 'series:"anything"'.

If you right-click a collection of files and choose "open externally" or "share->open->in file browser" you'll get a FileMissingException: FileMissingException
No file found at path + C:\Users\ANONYM~1\HYDRUS~1.ONL\HYDRUS~1\db\client_files\f9e\9e9129364271ee790f1a7cd078ee3f8335ff76bee768f42f0f6b971ab301ee04.collection!
File "include\ClientGUIMenus.py", line 122, in event_callable
callable( *args, **kwargs )
File "include\ClientGUIMedia.py", line 947, in _OpenFileLocation
path = client_files_manager.GetFilePath( hash, mime )
File "include\ClientCaches.py", line 995, in GetFilePath
return self.LocklessGetFilePath( hash, mime )
File "include\ClientCaches.py", line 1057, in LocklessGetFilePath
raise HydrusExceptions.FileMissingException( 'No file found at path + ' + path + '!' )

not really a bug anyone will find unless you're adding your own mods to the source in thread watcher page -> tag import options: the default namespaces (i.e. filename checkbox) are hardcoded and are not being pulled from ClientDefaults.GetDefaultNamespacesAndSearchValue() where as in file -> options -> default tag options -> add/edit -> thread watcher: the namespaces are pulled from the defaults to repro: edit ClientDefaults.py line 372 and add another namespace e.g.: namespaces = [ 'filename', 'title' ] file -> options -> default tag options -> add/edit -> thread watcher : there are two checkboxes (filename and title) thread watcher page -> tag import options : only one checkbox (filename), should be two From 320573c024a8a3366df802a72942a844471a1a45 Mon Sep 17 00:00:00 2001
From: Anonymous <anon@anon.com>
Date: Tue, 26 Dec 2017 01:08:38 +1100
Subject: [PATCH] Unhardcoded namespace list in tag import options popup on the
thread watcher page

---
include/ClientGUIManagement.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/include/ClientGUIManagement.py b/include/ClientGUIManagement.py
index f0095f8..5d57742 100755
--- a/include/ClientGUIManagement.py
+++ b/include/ClientGUIManagement.py
@@ -2080,8 +2080,10 @@ class ManagementPanelImporterThreadWatcher( ManagementPanelImporter ):

( thread_url, file_import_options, tag_import_options ) = self._thread_watcher_import.GetOptions()

+ ( namespaces, search_value ) = ClientDefaults.GetDefaultNamespacesAndSearchValue( ClientDownloading.GalleryIdentifier( HC.SITE_TYPE_THREAD_WATCHER ) )
+
self._file_import_options = ClientGUIImport.FileImportOptionsButton( self._thread_watcher_panel, file_import_options, self._thread_watcher_import.SetFileImportOptions )
- self._tag_import_options = ClientGUIImport.TagImportOptionsButton( self._thread_watcher_panel, [ 'filename' ], tag_import_options, self._thread_watcher_import.SetTagImportOptions )
+ self._tag_import_options = ClientGUIImport.TagImportOptionsButton( self._thread_watcher_panel, namespaces, tag_import_options, self._thread_watcher_import.SetTagImportOptions )

#

--
2.8.0.windows.1 i dont know if devbro likes patches or not but i find them easier for communicating and i hope it makes it easier for devbro to write his own patch

>>7559 tried running database -> check -> file integrity and seeing if any of your files are actually missing?

Got some sort of weird bug here. Might be unrelated but previous database checks have shown I was missing a file, but I had no idea how to find it in the db. So today I import and following a speculative similar search I get this file which I can't delete/inbox/archive or open in the media viewer, but I am able to modify tags. Unfortunately, searching for any unique tags I add to that file in order to keep track of it acts as if those tags don't exist. I know the system sort of acts like this if you search across all known files (so files that may not actually be in your db) but this only happens for this one search for this one file, and it is still set to only search my files. Not sure how to go about removing this thing.

2018/01/06 11:18:08: Exception:
2018/01/06 11:18:08: AttributeError
'NoneType' object has no attribute 'shape'
Traceback (most recent call last):
File "/opt/hydrus/include/HydrusThreading.py", line 242, in run
callable( *args, **kwargs )
File "/opt/hydrus/include/ClientRendering.py", line 402, in THREADRender
frame = GenerateHydrusBitmapFromNumPyImage( numpy_image, compressed = False )
File "/opt/hydrus/include/ClientRendering.py", line 25, in GenerateHydrusBitmapFromNumPyImage
( y, x, depth ) = numpy_image.shape
AttributeError: 'NoneType' object has no attribute 'shape'
when trying to open one webm

The default regex for "E:\my collection\author name - v4c1p0074.jpg -> author name" seems to be broken, at least in v287 >[^\\]+*(?=\s-) Gives a "multiple result" error, and regexr.com just says error.

>>7519 I am always overwhelmed with things to do, so a lot of my code has been thrown together just to get something going. I don't always have great ui behaviour when things go wrong. It is a long-term project to rewrite older parts of the program to behave more neatly and put potentially blocking sections like this on their own thread. Thank you for reporting this–I will make a job to put some time into it. >>7542 Thank you for this report. This is an interesting problem–a lot of the siblings and parent code is still first-version prototype and is basically magic tricks to make you think at the gui level that tags are one thing while the db retains the original truth of what they are. as a result, some db-level search code doesn't have a special hook to compensate. I will check the *anything* search code out to see if I can slip in easy siblings support, but if it is super complicated, it will have to wait for the next iteration over the whole siblings and parents system, which will likely extend sibling and parent awareness to a new db cache layer that can be efficiently searched without a bunch of CPU checking these links. >>7559 Thank you for this report. This is fixed for v289–let me know if it gives you any more trouble. >>7561 Thanks for this interesting report. I have done exactly as you suggest. The whole shebang will be replaced in the downloader overhaul, with the namespace list being populated by what the parsing scripts say they support and everything running off the same system, so this will all disappear anyway in a bit. It sounds like you'll be well able to create your own parsing scripts when all that happens, so let me know if the new thread watcher parsing stuff doesn't do what you would like as I roll it out. >>7599 Thank you for this report. There seems to be some kind of problem syncing files being deleted and similar file information in the duplicate search tree. Some other users have noticed this same 'file is remote, but shouldn't be seen in this local-file-only search context' after doing some dupe filtering. I think I have identified how these spurious results are appearing, if not the cause. It might have been old code that I more recently fixed, so some users just have some orphans hanging around. In any case, running database->regen->similar files search tree in v289 should clear these bad results out. Please run it when convenient and let me know if this search still produces the bad result, and if these bad results reappear at all. >>7634 Thank you. It looks like the client is having trouble rendering the first frame of that video, which is unexpected as it must have imported ok at one point, which would have needed the first frame to generate the thumbnail. I have improved this error text, but can you post/link/send me the webm so I can check it out on my end? If you go help->about, what version of ffmpeg do you appear to have? >>7642 Thank you. That looks like a typo. I updated some of this stuff recently, so I think I must have copied something wrong. I have corrected the default for new users, but you will have to fix it yourself with the 'manage favourites' part. I think this will do the job: [^\\]+(?=\s-) i.e. Just take out the asterisk and you should be good.

So I took a pause from hydrus, due to moving computers (Debian to Antergos/Arch). Today I decided I wanted to migrate, and obviously I run into a heap of bugs. Stuff I've noticed while attempting to fix my database: - Crash on launch when one or two of client.mappings.db, client.master.db or client.db are missing. An error message would be nice. - The database assumes thumbnails being in a specific location, when the custom thumbnail locations are specified. I needed to set all locations in client_file_locations to "client_files", and overwrite my options value, before it would work. - The cache is awfully fragile. Some cache tables are not automatically regenerated if not found. Some can be manually created in the database menu, but at least one of them (shape_vptree) are deadlocked. (See below) - I get notifications about some "BetterListCtrl" being deleted now and then. Not sure what that is about. (Again see below)
Traceback (most recent call last):
File "include/HydrusDB.py", line 527, in _ProcessJob
result = self._Write( action, *args, **kwargs )
File "include/ClientDB.py", line 10699, in _Write
elif action == 'regenerate_similar_files': result = self._CacheSimilarFilesRegenerateTree( *args, **kwargs )
File "include/ClientDB.py", line 1835, in _CacheSimilarFilesRegenerateTree
self._c.execute( 'DELETE FROM shape_vptree;' )
OperationalError: no such table: shape_vptree

RuntimeError
wrapped C/C++ object of type BetterListCtrl has been deleted
File "include/ClientGUIListCtrl.py", line 1193, in EventContentChanged
self._UpdateButtons()
File "include/ClientGUIListCtrl.py", line 1132, in _UpdateButtons
if enabled_check_func():
File "include/ClientGUIScrolledPanelsReview.py", line 1349, in _FileLocationSelected
locations = self._current_media_locations_listctrl.GetData( only_selected = True )
File "include/ClientGUIListCtrl.py", line 807, in GetData
indices = self._GetSelected()
File "include/ClientGUIListCtrl.py", line 593, in _GetSelected
i = self.GetFirstSelected()
File "wx/core.py", line 2715, in _ListCtrl_GetFirstSelected
File "wx/core.py", line 2722, in _ListCtrl_GetNextSelected
Good luck. Love your work.

>>7672 I fixed the cache issue and the BetterListCtrl issue by creating an entirely new database, and copying the fresh client.caches.db over into my old database. I realize that deleting the client.caches.db is pretty fucked up, but would be pretty nice if a fresh one was automatically generated. (Or a button somewhere to do it.)

>>7665 >I think I have identified how these spurious results are appearing, if not the cause. It might have been old code that I more recently fixed, so some users just have some orphans hanging around. In any case, running database->regen->similar files search tree in v289 should clear these bad results out. Please run it when convenient and let me know if this search still produces the bad result, and if these bad results reappear at all. No dice, still shows up. Not a big deal, I was just worried my db was screwed up somehow

>>7672 Thank you for this info. I have saved it and will see about improving cache regen and so on when I next work on db recovery stuff. Just for my general info: Do you happen to remember approx what version you were updating from? Are we talking a ten week gap or a year+ gap? Did you go in one jump, just installing the latest version, or did you do smaller hops? Did that overall update process work ok? The BetterListCtrl stuff is a bug from this week's v288, which involved a big ui update. I have it fixed for tomorrow. >>7675 Thanks. Please try again in tomorrow's v289 release whenever is convenient, and I think you'll be good.

>>7682 I think it was 10-15 weeks worth of versions, I updated all at once, and it looked like it worked nicely.

?? on managing a file's tags the "file lookup scripts" clicking fetch tags i get TypeError TIMERUIUpdate() takes exactly 2 arguments (1 given) Traceback (most recent call last): File "include\ClientGUI.py", line 3239, in TIMEREventUIUpdate window.TIMERUIUpdate() TypeError: TIMERUIUpdate() takes exactly 2 arguments (1 given)

>>7692 Thank you, this should be fixed in today's release.

When my disk is full, and I got error, because hydrus cannot save the file on the disk, and then I move some hydrus files to another disk(with bultin function) and redownload not saved files only thumbnails are viewable. I'm getting errors like this one: IOError: cannot identify image file <open file u'/path/hydrus/fff/ffd911b8b2e44a0d94f6463ac0084f929dc6cccd2ab7f91d96372ed887889376.png', mode 'rb' at 0x7f4fd3b29db0> I'm deleting and redownloading files as an avoidance, but it's pain in the ass.

>>7694 If you are lucky, you can end with half downloaded, broken image. If you are not, the image will not load.

2018/01/12 22:26:05: hydrus client started
2018/01/12 22:26:13: booting controller...
2018/01/12 22:26:13: booting db...
2018/01/12 22:26:13: preparing disk cache
2018/01/12 22:26:18: preparing db caches
2018/01/12 22:26:18: booting db...
2018/01/12 22:26:18: initialising managers
2018/01/12 22:26:38: booting gui...
2018/01/13 00:21:22: shutting down gui...
2018/01/13 00:21:22: waiting for daemons to exit
2018/01/13 00:21:23: vacuuming main
2018/01/13 00:21:25: Vacuumed /home/user/hydrus-289/db/client.db in 1.7 seconds
2018/01/13 00:21:25: vacuuming external_master
[1] 20928 segmentation fault (core dumped) ./client.pyw 2> /dev/null Hydrus crashes on shut down when doing maintenance. If I do the vacuuming manually, it doesn't crash anymore. Running from source on Linux.

>>7695 >>7694 Thank you for this report. I will make a job to look into better error handling for broken files, maybe some kind of popup with a question and possible solutions like 'retry download' or 'delete completely' rather than always making the user do it. >>7713 Thank you, I will look into this.

Could not create a thumbnail from that video!… (Copy note to see full error)
Traceback (most recent call last):
File "include\ClientImporting.py", line 4135, in _WorkOnFiles
( status, hash ) = HG.client_controller.client_files_manager.ImportFile( file_import_job )
File "include\ClientCaches.py", line 1007, in ImportFile
file_import_job.GenerateInfo()
File "include\ClientImporting.py", line 395, in GenerateInfo
self._thumbnail = HydrusFileHandling.GenerateThumbnail( self._temp_path, mime )
File "include\HydrusFileHandling.py", line 119, in GenerateThumbnail
raise Exception( 'Could not create a thumbnail from that video!' )
Exception: Could not create a thumbnail from that video!
When importing this file: https://chan.sankakucomplex.com/post/show/3897585

why can't i get comic pages sorted by page number? im using the "page:##" tag and the "title:xxxxxx" tag for all pages, the sort is all over the fucking place, some of the pages go in the right position, but others go wherever the fuck they please, even when i change the "sort by-" option, the result is that no matter what type of sort i end up with shit like 1, 2, 3, 4, 6, 8, 10, 11, 5, 7, 9 instead of proper numerical order

Just got a "program is not responding. reason: APPCRASH" when resuming Windows from hibernation. Then after force-closing it through windows, the window stuck around like this for a few seconds. This is all that's in client.log since yesterday. Doesn't look like anything relevant: 2018/01/18 21:10:21: Import folder import folder imported 1 files. 2018/01/19 02:10:27: Import folder import folder imported 1 files. 2018/01/19 10:37:06: hydrus client started 2018/01/19 10:37:06: booting controller… 2018/01/19 10:37:06: booting db… 2018/01/19 10:37:06: preparing disk cache 2018/01/19 10:37:11: preparing db caches 2018/01/19 10:37:11: booting db… 2018/01/19 10:37:11: initialising managers 2018/01/19 10:37:33: booting gui…

Attempting to import files (two pngs and a webm) from one particular folder produced this error: WindowsError [Error 6] The handle is invalid Traceback (most recent call last): File "include\HydrusThreading.py", line 242, in run callable( *args, **kwargs ) File "include\ClientGUIDialogs.py", line 1148, in THREADParseImportablePaths mime = HydrusFileHandling.GetMime( path ) File "include\HydrusFileHandling.py", line 275, in GetMime if HydrusVideoHandling.HasVideoStream( path ): File "include\HydrusVideoHandling.py", line 377, in HasVideoStream lines = GetFFMPEGInfoLines( path ) File "include\HydrusVideoHandling.py", line 141, in GetFFMPEGInfoLines proc = subprocess.Popen( cmd, bufsize = 10**5, stdout = subprocess.PIPE, stderr = subprocess.PIPE, startupinfo = HydrusData.GetHideTerminalSubprocessStartupInfo() ) File "subprocess.py", line 382, in init File "subprocess.py", line 515, in _get_handles File "subprocess.py", line 566, in _make_inheritable WindowsError: [Error 6] The handle is invalid Trying to import them again produced the same error message again Restarting the client fixed it and I then imported them succesfully The webm was pic related

>>7733 Thank you. I get the same problem with ffmpeg 3.4.1. It looks like ffmpeg is dumping out before delivering the first frame, which is what it usually does whenever it can't understand something. What's odd is I was able to convert the file to a webm (attached), just by simply going: ffmpeg a.webm -i a.png And it isn't all corrupted or anything. I am not sure why ffmpeg can't handle converting this apng to a raw stream as the client wants (my other apngs are fine, so I am pretty confident the pipeline on my end is good), but I recommend either: 1. Keep the apng somewhere safe and try it again later. If this is a bug in ffmpeg, it may just magically work one day after I or you update the .exe in the build. 2. Use my webm here or make a better conversion to whatever standard and quality you like. Or both. EDIT: After looking closer at my webm, I see it is bad quality. I tried rendering it with higher quality, but ffmpeg always delivered the same 100KB washy mess. Maybe this is due to the same original weirdness that is causing the other render problem. If you know ffmpeg yourself or have a different conversion solution, you may have more luck. I recommend keeping this apng in a drawer for now. Thank you for the example–I will keep it in my 'busted files' folder and check up on it in future. >>7734 Thank you for this report. I am afraid I cannot reproduce it. The 'sort by namespace' choice sorts from left to right, and my series-creator-title-volume-chapter-page and creator-series-title-volume-chapter-page defaults, 'title' comes before 'page'. This gives good page sort if every page in a group has the same chapter title of, say, 'title:going to the store', but if all your pages have different titles they will be sorted by title first. If you go to the options->sort/collect page (which is a user-interface hellscape, I apologise), try typing 'page' at the bottom and hitting enter. All new pages should then have the single 'sort by page' dropdown entry. If you try this simple sort, does it work then? If you commonly have paged files with individual titles, you might want to try adding a creator-series-volume-chapter-page sort type. >>7755 Thank you for this report. This may have been one of the existing ui crashes that I am working on cleaning up, which could trigger on a sleep/wake event as certain objects are cleaned up. I apologise for the inconvenience. Please let me know if you continue to get this crash in future versions. >>7773 And thank you for this report as well. I have not seen this before. It looks like the client was unable to launch ffmpeg (a video renderer it uses for apng detection and webm parsing) due to some OS issue. Basically, it says "Hey Windows, can you open ffmpeg.exe for me with these args?" and Windows says "No." for an unspecified reason. A bit of searching suggests this can be caused by overzealous anti-virus. Could this be true in your case? Since it worked after a restart, maybe your client's process got locked or deprivileged or something while a scan was ongoing? Since hydrus is ghetto code, false positives of this sort are not uncommon, but I can't remember seeing the client exe able to boot but then unable to launch an exe in its subdir before. If it happens again, please shift+right-click on your install_dir/bin folder and hit 'open command window here'. Then type 'ffmpeg' and hit enter (you might have to do '.\ffmpeg' if your Windows launches the powershell). Does it print a bunch of info about ffmpeg or another 'you can't talk to this right now' kind of error?

>>7808 >>7733 Ah, that webm doesn't render in firefox. It was ok-ish in MPC. I guess that apng is just messed up and ffmpeg has a hard time dealing with it.

>>7809 it don't work in chrome either lol

>>7809 >>7813 And it insta crashes vlc

>>7808 >>7809 >>7813 >>7814 To preserve quality with webms it's best to use the `-crf` flag in ffmpeg it attempts to keep a constrained quality through the file. ffmpeg also sucks at handling apngs, this somewhat has to do with the fact that ffmpeg is meant for manipulating videos which have some sort of constant framerate whether it be 24fps or 30 or what have you. apng (and gif) works along the line of play this frame and then delay for 1s and then play this frame and delay for 0.1s leading to some loss of information when converting. Here I just guesstimated the framerate and told ffmpeg that the input file is of that framerate, that's the `-r` flag, hence why it's before the `-i` flag. This still mangles the apng in some way as there was a 1.25s delay at the start of the file that gets left out due to this constant framerate. ffmpeg -r 12 -i a.png -crf 31 -b:v 0 a.webm The webm doesn't play in firefox because ffmpeg defaults to the VP9 codec which apparently firefox doesn't support yet? You can force using VP8 using `-c:v vp8`. ffmpeg -r 12 -i a.png -crf 31 -b:v 0 -c:v vp8 a.webm In other words, fuck video and "animated images".

My client doesn't boot :( I tried a new install but the error is the same. Joined here is the client_debug.

I've encountered a bug with the danbooru parser today: The parser will automatically stop if it can't find any image on a page, but since default danbooru searches have blank placeholder items for deleted posts, it's possible that all images on a page are deleted and the parser will stop despite there being pages left. And that's exactly what happened to me. Of course, this bug can be prevented by adding "-status:deleted" to the search query for now, but without it you might encounter this bug in any query. Furthermore, banned posts might be affected, too. And while "-status:deleted" doesn't count towards the maximum of two tags, "-status:banned" does.

>>7818 Thank you for this information. Maybe some experts in the industry should get together and make a single future-proof standard that would cover all these unusual situations? :^) >>7896 Hey, I am sorry you are having trouble. Have you been able to run hydrus before, or has this error just started with this version? This error means one of the libraries I use is having trouble importing some dll or similar from your system. I looked at the line it is complaining about, and it looks like it bugged out at a character in your PATH or home directory. I've seen a similar problem to this before a few times with a different library. Do you have a unicode or unusual accent character in your username? Is your home directory like: C:\users\(name with unusual character) Or do you have any unusual characters in your PATH environment variable? (search this term to see how to look it up, if you do not know–but do not edit it unless you know exactly what you are doing!) If this is the case and you know you can change or remove the offending path without breaking anything, that is the fix. If you cannot change it, unfortunately python2 cannot deal with this situation, so you would have to put hydrus on a different computer to get it to run. I expect to update to python3, which does not have this limitation, sometime this year. >>7919 Thank you, this is useful and interesting. There is not a good fix for this in the current downloading engine, but the new one that I am currently putting together will explicitly look for 'next page' links and will be able to detect 'no results found' results better than just counting up what it finds. I will keep this danbooru stuff in mind as I put it together. What is the point of displaying deleted and banned placeholders to users? Does danbooru allow conversation about ban decisions or something, or is it just some cost-saving measure so they don't have to regen their caches?

>>7923 >What is the point of displaying deleted and banned placeholders to users? A bit confusing, but "deleted" posts are actually just hidden, so the placeholders are there to show that some of the posts on the page are hidden. You can still browse through "deleted" posts by adding "status:deleted" to your query and it doesn't even count towards the maximum of two tags. It's also possible to appeal a deletion. I guess they only actually delete submissions if the content is illegal. But there are two other groups of content that only show a placeholder: Censored posts and banned posts. Censored posts include the loli, shota and/or toddlercon tags and can still be viewed with a gold account (or as a "Builder"). Banned posts are posts by banned artists (artists who requested their art to be removed). This is the only group of hidden posts that actually seems to be gone for good. I can think of a few reasons why they would keep the placeholders: The possibility that the artist changes their mind or just as a notification, so that the user could try to get it from the official source. In any case, at least the tags of banned posts are still there for some reason. There's also possibly a fourth group: Some blacklisted tags are toggled by default (e.g. the "spoilers" tag), but I'm not sure if this affects the parser.

Crash report. client - 2018-2.log contains nothing since I updated to 292 yesterday. Hydrus was just sitting in the background and I wasn't interacting with it, though a minute or so earlier I had added tags to some files in the inbox.

>>7923 You were right, moving the folder to a location without unicode fixed the issue and helped me boot. And yes, it worked before but I hadn't used Hydrus in a few months. Sadly, I've got another issue on my hands now. I was able to use Hydrus once, but I can not start it again. This is the issue I get when trying to use Hydrus. Looks like the error is with my database, I think it closed normally the last time I used Hydrus, but I might be wrong.

>>7808 >>7733 >>7813 >>7814 >>7818 Ok, I fixed this for tomorrow's release. Turns out ffmpeg can't do "-ss 0.000" (i.e. 'scan to the beginning') for some apngs. I've removed this redundant parameter and the import now works. For interest, the typical ffmpeg call hydrus makes is: [ffmpeg path] -ss 0.000 -i [file path] -loglevel quiet -f image2pipe -pix_fmt rgb24 -s 200x200 -vsync 0 -vcodec rawvideo - Essentially, 'gimme that video as raw RGB to stdout, thanks.' The 200x200 size here is for the initial thumbnail generation.

I can no longer open Options. The only thing I changed was allowing all files to be automatically archived when imported. Here's the traceback info:
ValueError
need more than 3 values to unpack
File "include\ClientGUIMenus.py", line 141, in event_callable
callable( *args, **kwargs )
File "include\ClientGUI.py", line 1940, in _ManageOptions
panel = ClientGUIScrolledPanelsManagement.ManageOptionsPanel( dlg )
File "include\ClientGUIScrolledPanelsManagement.py", line 1310, in __init__
self._listbook.AddPage( 'default file system predicates', 'default file system predicates', self._DefaultFileSystemPredicatesPanel( self._listbook, self._new_options ) )
File "include\ClientGUIScrolledPanelsManagement.py", line 2006, in __init__
self._file_system_predicate_age = ClientGUIPredicates.PanelPredicateSystemAgeDelta( self )
File "include\ClientGUIPredicates.py", line 92, in __init__
( sign, years, months, days, hours ) = system_predicates[ 'age' ]


>>7939 There's another manage tags dialog crash fix in the release I am putting out later today. I hope it fixes this problem, but if you still get more, please let me know. >>7945 I am not sure what is going on here. I expect I am too late as this problem has fixed itself, but if it is still going on, I expect the old hydrus program instance did not shut down correctly. Maybe it was doing a vacuum or something and somehow closed the splash screen as well? In any case, please open task manager (Ctrl+Shift+Esc in Windows) and look down the app list for any lingering instances of client.exe. Kill them (just select and hit delete->ok) and you should be good to try again. If it is still having a problem, what happens if you reboot your computer and start fresh? If it is still having a problem then, what happens if you extract a fresh client on your desktop and run it? Does that boot ok the first time, and does it go ok if you restart it? I don't think that the previous unicode issue would lead to subsequent boot issues, and even if it did, I would not expect it to hang or cause a deadlock on boot, so I think at the moment that this is a temporary thing that the above steps will figure out if it hasn't already fixed itself. If it isn't, let me know and we'll drill deeper.

>>7956 I see that you potentially fixed my problem in the newest update tomorrow, so thank you. Also, my client is crashing randomly. I don't lose much data, maybe the tags on the image I was working on, but it is a bit annoying. Dunno how to find a log for it either. And whenever I make a keystroke, I'm getting a "key event caught" with a notification appearing in the bottom right. Weh.

The file import folder seems to consistently ignore files named "Untitled.png". If I crop a panel from a manga or something I will usually just save it into the import folder with the default filename (Untitled.png), since the filename will stop mattering as soon as the file gets imported anyways. Some time in the past couple of weeks this has stopped working. Files with other names still get imported automatically, even if I give them the same name as another file that was imported recently. But anything named Untitled.png gets ignored, even if I go into Hydrus and click "check import folder now". I end up having to import the file manually (or just rename it).

Video skip stopped working and now just pretends earlier frames are later frames. GIF and PNG skips still work as normal

>>5115 About half the time I run a duplicates discovery, the client crashes. I ran it in-terminal and caused the crash as fast as I could, which took two attempts (exact then very similar). this is my whole output: WARNING:root:pafy: youtube-dl not found; falling back to internal backend. This is not as well maintained as the youtube-dl backend. To hide this message, set the environmental variable PAFY_BACKEND to "internal".
2018/02/09 05:46:51: hydrus client started
05:46:51 AM: Debug: Failed to connect to session manager: SESSION_MANAGER environment variable not defined
2018/02/09 05:46:51: booting controller...
2018/02/09 05:46:51: booting db...
2018/02/09 05:46:51: preparing disk cache
2018/02/09 05:46:51: preparing db caches
2018/02/09 05:46:51: booting db...
2018/02/09 05:46:51: initialising managers
2018/02/09 05:46:52: booting gui...

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkScrollbar

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton

(client.pyw:20068): Gtk-CRITICAL **: gtk_box_gadget_distribute: assertion 'size >= 0' failed in GtkSpinButton
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python2: xcb_io.c:259: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.
[1] 20068 abort (core dumped) hydrus-client

>>7958 Thank you for this report. I apologise for the crashes–some came in when I updated wx around v288. I am improving the situation but there is more to do. I think the 'key event' stuff is from help->debug->report modes->gui report mode. Could that have been switched on by accident? Or are you maybe using a debug build I put together for someone that might have had that on by default? This mode should not be on when you start the client–if it is for you, what happens if you try to click it again, does it turn off? >>7990 Thank you for this report. I pushed it too hard last week and made a stupid mistake here. This is fixed for v294. >>7991 Thank you for this report. I had a similar report from someone else, also on Linux. I will check this this week and see if I can figure it out.

>>8000 I'll have to update it to the newest version to open the options log soon, but I'll let you know. The 'key event' thing went away after I restarted my computer so that's all good now. Thanks for getting back to me.

Hydrus said something about a "serious error" on startup where it wasn't able to initialize the db or something. The error said it wrote the traceback to the log so I closed it and checked the latest log file but couldn't find anything. The error didn't happen on my next attempt so I don't have a traceback for you

>>8009 Never mind I found it 2018/02/11 14:02:37: A serious error occured while trying to start the program. Its traceback will be shown next. It should have also been written to client.log.
2018/02/11 14:02:37: Traceback (most recent call last):
File "include\ClientController.py", line 1204, in THREADBootEverything
self.InitModel()
File "include\ClientController.py", line 496, in InitModel
HydrusController.HydrusController.InitModel( self )
File "include\HydrusController.py", line 297, in InitModel
self.db = self._InitDB()
File "include\ClientController.py", line 84, in _InitDB
return ClientDB.DB( self, self.db_dir, 'client', no_wal = self._no_wal )
File "include\ClientDB.py", line 153, in __init__
HydrusDB.HydrusDB.__init__( self, controller, db_dir, db_name, no_wal = no_wal )
File "include\HydrusDB.py", line 241, in __init__
raise Exception( 'Could not initialise the db! Error written to the log!' )
Exception: Could not initialise the db! Error written to the log!

Too many crashes. high cpu use, or high ram, and now disk thrashing when it locks up.

the manage tags dialog still giving trouble I don't know if this warning was related to the crash but maybe it is useful

>>8014 OS: Windows 10 All I did as open it up, then quickly open and close the manage tags dialog to get something to happen

>>7939 Same again. I had been adding tags, then I opened an image up in an external viewer. A few seconds later, while it's in the background behind the external image viewer, hydrus crashes

>>8007 There should be an important improvement in the crashing in tomorrow's release, btw. Please check it out! >>8009 >>8010 Hey, I am sorry, but is there another error just above that one in the log? That should be the one that is causing the problem–the one in >>8010 is just the notification and a higher level of the program reeling from the db not booting properly. If your client now boots ok, this may not be a huge worry–maybe your disk was locked or something from a previous version of the client closing down, or your defragger/anti-virus was stomping on things? In any case, see if there is another traceback and let me know, so we can make sure. I will see if I can get the db-level error to come at you in an emergency window in tomorrow's release, so if it happens again you'll have something you can screenshot easier. >>8011 >>8014 >>8015 >>8021 Thank you for these reports. Tomorrow's release should have a final fix to the manage tags dialog–I have removed the new-wx crashy timer objects entirely and replaced the functionality with a new pure-python thread scheduler I have written. This new scheduler will reduce idle CPU and thread count across the program, particularly in the coming weeks. I've been pushing it a bit hard in recent weeks trying to get the downloader overhaul going, and I have been making mistakes and haven't dealt with this crashy and bloaty behaviour as quickly as I should have. I will be more careful in allocating my time in the future and try to get this shit fixed as a high priority going forwards. Please let me know how v294 goes for you, and I apologise again. >>8024 Thank you for this report. I am afraid this file imports and plays ok for me. If you hit help->about, what version of ffmpeg does your client think it has? My Windows releases currently come with 3.4.1–are you significantly older than that? Linux and OS X builds try to pick up what the system provides, so a good fix here is either to get your system to update its natural ffmpeg or to chase down a newer static built ffmpeg executable and drop it in hydrus's install_dir/bin directory. If your ffmpeg seems ok, please rename that file to something simple like 'a.mp4' and try this in a terminal: ffmpeg -i a.mp4 This will cause ffmpeg to dump a bunch of info it can figure about the file to the terminal. Does it seem encouraging, or does it have errors?

>>7991 Getting a similar crash. Here's a backtrace. #0 0x00007f5956cac0c4 in wxEvtHandler::SafelyProcessEvent(wxEvent&) ()
at /usr/lib/python2.7/site-packages/wx/libwx_baseu-3.0.so.0
#1 0x00007f5956c12561 in wxTimerImpl::SendEvent() ()
at /usr/lib/python2.7/site-packages/wx/libwx_baseu-3.0.so.0
#2 0x00007f59580d2241 in sipwxTimer::Notify() () at /usr/lib/python2.7/site-packages/wx/_core.so
#3 0x00007f5957414491 in () at /usr/lib/python2.7/site-packages/wx/libwx_gtk3u_core-3.0.so.0
#4 0x00007f5959f3c783 in () at /usr/lib/libglib-2.0.so.0
#5 0x00007f5959f3bca6 in g_main_context_dispatch () at /usr/lib/libglib-2.0.so.0
#6 0x00007f5959f3c081 in () at /usr/lib/libglib-2.0.so.0
#7 0x00007f5959f3c3b2 in g_main_loop_run () at /usr/lib/libglib-2.0.so.0
#8 0x00007f59563dbd2f in gtk_main () at /usr/lib/libgtk-3.so.0
#9 0x00007f5957402675 in wxGUIEventLoop::DoRun() ()
at /usr/lib/python2.7/site-packages/wx/libwx_gtk3u_core-3.0.so.0
#10 0x00007f5956b6c373 in wxEventLoopBase::Run() ()
at /usr/lib/python2.7/site-packages/wx/libwx_baseu-3.0.so.0
#11 0x00007f5956b346b6 in wxAppConsoleBase::MainLoop() ()
at /usr/lib/python2.7/site-packages/wx/libwx_baseu-3.0.so.0
#12 0x00007f5957fce5b1 in wxPyApp::MainLoop() () at /usr/lib/python2.7/site-packages/wx/_core.so
#13 0x00007f5957fce7da in () at /usr/lib/python2.7/site-packages/wx/_core.so
#14 0x00007f596516de39 in PyEval_EvalFrameEx () at /usr/lib/libpython2.7.so.1.0
#15 0x00007f596516e550 in PyEval_EvalFrameEx () at /usr/lib/libpython2.7.so.1.0
#16 0x00007f596516e550 in PyEval_EvalFrameEx () at /usr/lib/libpython2.7.so.1.0
#17 0x00007f59651c8456 in PyEval_EvalCodeEx () at /usr/lib/libpython2.7.so.1.0
#18 0x00007f59651e1a9a in PyEval_EvalCode () at /usr/lib/libpython2.7.so.1.0
#19 0x00007f59651e5321 in run_mod () at /usr/lib/libpython2.7.so.1.0
#20 0x00007f59651e6c05 in PyRun_FileExFlags () at /usr/lib/libpython2.7.so.1.0
#21 0x00007f59651e6dda in PyRun_SimpleFileExFlags () at /usr/lib/libpython2.7.so.1.0
#22 0x00007f59651ce3b3 in Py_Main () at /usr/lib/libpython2.7.so.1.0
#23 0x00007f59654e6f4a in __libc_start_main () at /usr/lib/libc.so.6
#24 0x000055c090da678a in _start ()


2018/02/14 10:57:04: hydrus client started 2018/02/14 10:57:04: booting controller… 2018/02/14 10:57:04: booting db… 2018/02/14 10:57:04: preparing disk cache 2018/02/14 10:57:09: preparing db caches 2018/02/14 10:57:09: booting db… 2018/02/14 10:57:09: initialising managers 2018/02/14 10:57:45: booting gui… 2018/02/14 10:58:19: Import folder import folder imported 1 files. 2018/02/14 10:58:33: Uncaught exception: 2018/02/14 10:58:33: RuntimeError wrapped C/C++ object of type MediaPanelThumbnails has been deleted File "include\ClientGUIMedia.py", line 3592, in EventShowMenu HG.client_controller.PopupMenu( self, menu ) File "include\ClientController.py", line 849, in PopupMenu ClientGUIMenus.DestroyMenu( menu ) File "include\ClientGUIMenus.py", line 121, in DestroyMenu UnbindMenuItems( menu ) File "include\ClientGUIMenus.py", line 102, in UnbindMenuItems event_handler.Unbind( wx.EVT_MENU, source = menu_item ) File "site-packages\wx\core.py", line 1353, in _EvtHandler_Unbind File "site-packages\wx\core.py", line 1422, in Unbind Just after starting up I right-clicked and picked "refresh" in the main window (system:everything order:random), and this popped up at the same time as the newly-refreshed thumbnails became visible.

On version 294, received this error twice while running the client in the background: RuntimeError wrapped C/C++ object of type AutoCompleteDropdownTagsRead has been deleted File "include\ClientGUIACDropdown.py", line 500, in EventMove if self._ShouldShow(): File "include\ClientGUIACDropdown.py", line 223, in _ShouldShow tlp_active = self.GetTopLevelParent().IsActive() or self._dropdown_window.IsActive()

>>8026 A'ight. Right before that error there's this 2018/02/11 14:02:37: Traceback (most recent call last):
File "include\HydrusDB.py", line 750, in MainLoop
self._InitDiskCache()
File "include\ClientDB.py", line 6621, in _InitDiskCache
self._LoadIntoDiskCache( stop_time = stop_time )
File "include\ClientDB.py", line 6738, in _LoadIntoDiskCache
self._InitDBCursor()
File "include\HydrusDB.py", line 483, in _InitDBCursor
raise HydrusExceptions.DBAccessException( HydrusData.ToUnicode( e ) )
DBAccessException: database is locked

Hey Hdev, things seemed like they were going smoother with crashes in 294 it did freeze up at one point but came back alive. Well now i'm tossing in another bug report it;l keep spamming RuntimeErrors. RuntimeError wrapped C/C++ object of type AutoCompleteDropdownTagsRead has been deleted File "include\ClientGUIACDropdown.py", line 500, in EventMove if self._ShouldShow(): File "include\ClientGUIACDropdown.py", line 223, in _ShouldShow tlp_active = self.GetTopLevelParent().IsActive() or self._dropdown_window.IsActive()

Left it idling for a couple hours and this showed up at some point. RuntimeError
wrapped C/C++ object of type AutoCompleteDropdownTagsRead has been deleted
File "include\ClientGUIACDropdown.py", line 500, in EventMove
if self._ShouldShow():
File "include\ClientGUIACDropdown.py", line 223, in _ShouldShow
tlp_active = self.GetTopLevelParent().IsActive() or self._dropdown_window.IsActive()


I get this error when trying to run Version 294, fresh 'extract only' client. OS is Win7. C:\Hydrus Network>client_debug.exe
PyInstaller Bootloader 3.x
LOADER: executable is C:\Hydrus Network\client_debug.exe
LOADER: homepath is C:\Hydrus Network
LOADER: _MEIPASS2 is NULL
LOADER: archivename is C:\Hydrus Network\client_debug.exe
LOADER: No need to extract files to run; setting extractionpath to homepath
LOADER: SetDllDirectory(C:\Hydrus Network)
LOADER: Already in the child - running user's code.
LOADER: Python library: C:\Hydrus Network\python27.dll
LOADER: Loaded functions from Python library.
LOADER: Manipulating environment (sys.path, sys.prefix)
LOADER: sys.prefix is C:\HYDRUS~2
LOADER: Setting runtime options
LOADER: Initializing python
LOADER: Overriding Python's sys.path
LOADER: Post-init sys.path is C:\Hydrus Network
LOADER: Setting sys.argv
LOADER: setting sys._MEIPASS
LOADER: importing modules from CArchive
LOADER: extracted struct
LOADER: callfunction returned...
LOADER: extracted pyimod01_os_path
LOADER: callfunction returned...
LOADER: extracted pyimod02_archive
LOADER: callfunction returned...
LOADER: extracted pyimod03_importers
LOADER: callfunction returned...
LOADER: Installing PYZ archive with Python modules.
LOADER: PYZ archive: out00-PYZ.pyz
LOADER: Running pyiboot01_bootstrap.py
LOADER: Running pyi_rth_twisted.py
Traceback (most recent call last):
File "site-packages\PyInstaller\loader\rthooks\pyi_rth_twisted.py", line 23, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\twisted\internet\default.py", line 16, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\twisted\python\__init__.py", line 11, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\twisted\python\compat.py", line 29, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "socket.py", line 47, in <module>
ImportError: No module named _socket
Failed to execute script pyi_rth_twisted
LOADER: OK.
LOADER: Cleaning up Python interpreter

>>8050 I got the same error. It seems to show up repeatedly when I minimize then restore the window.

>>8050 Update: I'm consistently getting this error every time I get a notification about a subscription downloading something, though I can't say if that's the only time it arises.

>>8050 Another guy with exactly this runtimerror reporting in. I don't have any subscriptions downloading anything though. I am consistently getting 1 or two popups with this same error message every time I perform certain actions >minimize and un-miminize (x2 error messages) >maximize (x1 error message) >un-maximize (x1) >drag window titlebar to move it (x2) >resize top or left border of window (x2) but not when I alt-tab or when I resize the bottom or right borders

>>8033 Thank you for this report. Was this in v294? I believe I have fixed the duplicate filter page crash in this latest release, but if you are still getting it, please let me know. >>8034 Thank you for this report. I haven't seen this one before, but I know what is going on, and I was just able to reproduce it. I will write in an extra layer of protection for next week. >>8038 >>8045 >>8050 >>8062 >>8063 >>8067 Thanks lads. I got this myself as well. This is from my crash fix. Thankfully, although it is annoying, it seems basically harmless. It looks like it is triggered by the manage tags dialog or a regular search page being closed. Restarting the client will clear the issue for now, and I will have a proper fix out for next week. >>8056 Thank you for this report. Do you happen to have an unusual character in your OS username (maybe an accent)? Or have you recently installed a core utility with a path like this, maybe some Japanese or Russian language-support or something? This error is python2 being unable to import a dll because your OS PATH includes a unicode or other unusual character. If you are unfamiliar with the PATH, a bit of searching will tell you how to look it up for Win7, but be extremely wary of editing it if you are not absolutely certain what you are removing is ok. Unfortunately, there is no easy fix for this my end. I expect to update to python3 this year, which will fix this issue for good.

When I export multiple files my client crashes. The files are exported successfully, but ever time I close the 'setup export' window the program crashes. I started the program once from the terminal and got this after the crash:
*** Error in `./client': double free or corruption (out): 0x00000000046ff940 ***
Aborted (core dumped)


>>8076 My username does have a character with a tilde. However, I have an old extract only client, version 212, and it works fine. What gives?

>>8094 Thank you for this report. I will check this out this week. >>8098 I am not sure–this is all happening at a lower level than I control. I think it might be that I am using a new or newer version of a library in the more recent version that is searching for its dlls in this way, whereas perhaps the older version used a different search? In your case, it is twisted (a networking library) that is breaking–maybe I didn't use that a couple years ago? Then again, looking closer at the exact line of code in socket.py that is failing, it doesn't look like it is using the same dll-search function I have seen before failing for unusual PATH users. This actually looks like it is the regular python 'import' phrase that isn't working. In your install folder, do you have "_socket.pyd"? It should be about 23KB. Could an over-zealous anti-virus program have blocked it for you?

>>8137 I realized that I didn't have all the files after unpacking the .rar for some reason. Having fixed that, I tried again (_socket.pyd is there) and now I get a slightly different error: C:\Hydrus Network>client_debug.exe
PyInstaller Bootloader 3.x
LOADER: executable is C:\Hydrus Network\client_debug.exe
LOADER: homepath is C:\Hydrus Network
LOADER: _MEIPASS2 is NULL
LOADER: archivename is C:\Hydrus Network\client_debug.exe
LOADER: No need to extract files to run; setting extractionpath to homepath
LOADER: SetDllDirectory(C:\Hydrus Network)
LOADER: Already in the child - running user's code.
LOADER: Python library: C:\Hydrus Network\python27.dll
LOADER: Loaded functions from Python library.
LOADER: Manipulating environment (sys.path, sys.prefix)
LOADER: sys.prefix is C:\HYDRUS~2
LOADER: Setting runtime options
LOADER: Initializing python
LOADER: Overriding Python's sys.path
LOADER: Post-init sys.path is C:\Hydrus Network
LOADER: Setting sys.argv
LOADER: setting sys._MEIPASS
LOADER: importing modules from CArchive
LOADER: extracted struct
LOADER: callfunction returned...
LOADER: extracted pyimod01_os_path
LOADER: callfunction returned...
LOADER: extracted pyimod02_archive
LOADER: callfunction returned...
LOADER: extracted pyimod03_importers
LOADER: callfunction returned...
LOADER: Installing PYZ archive with Python modules.
LOADER: PYZ archive: out00-PYZ.pyz
LOADER: Running pyiboot01_bootstrap.py
LOADER: Running pyi_rth_twisted.py
LOADER: Running pyi_rth_pkgres.py
LOADER: Running pyi_rth_win32comgenpy.py
LOADER: Running pyi_rth__tkinter.py
LOADER: Running pyi_rth_mplconfig.py
LOADER: Running pyi_rth_mpldata.py
LOADER: Running client.py
Traceback (most recent call last):
File "client.py", line 20, in <module>
from include import ClientController
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "include\ClientController.py", line 1, in <module>
import ClientCaches
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "include\ClientCaches.py", line 1, in <module>
import ClientDefaults
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "include\ClientDefaults.py", line 2, in <module>
import ClientData
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "include\ClientData.py", line 3, in <module>
import ClientDownloading
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "include\ClientDownloading.py", line 3, in <module>
import ClientNetworking
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "include\ClientNetworking.py", line 17, in <module>
import requests
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\requests\__init__.py", line 84, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\urllib3\contrib\pyopenssl.py", line 47, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\cryptography\x509\__init__.py", line 7, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\cryptography\x509\base.py", line 16, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\cryptography\x509\extensions.py", line 14, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\asn1crypto\keys.py", line 22, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\asn1crypto\_elliptic_curve.py", line 51, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\asn1crypto\_int.py", line 56, in <module>
File "c:\python27\Lib\site-packages\PyInstaller\loader\pyimod03_importers.py", line 389, in load_module
File "site-packages\asn1crypto\_perf\_big_num_ctypes.py", line 31, in <module>
File "ctypes\util.py", line 53, in find_library
File "ntpath.py", line 85, in join
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 14: ordinal not in range(128)

LOADER: OK.
LOADER: Cleaning up Python interpreter.

I can't download stuff from pixiv It just says "0 urls found for #id" no matter the id I'm using v296

>>8184 do you have a pixiv account setup?

>>8154 Right. I think you are now hitting the problem I first thought you had–a part of your PATH variable is not decodable to ASCII, so this python version's ctypes can't deal with it when it tries to find the right dll. On most later Windows, I think you can see the path if you go Win+Break (to open 'system') then hit 'advanced system settings'->environment variables->path. (You can search online if your Windows version doesn't have the same ui) I strongly recommend you not edit your PATH unless you know what you are doing, but if you review it, do you see any obvious candidates for bad entries? If it is something like C:\users\yõu\desktop\someprogramyouonceextractedbutneverusedagain, then we may be in luck. >>8184 I had to remove 'pixiv tags' search from modern releases recently when they changed their markup to a system I cannot parse with the current engine. I hope to have this fixed in the new parser. Afaik, pixiv artist search is still working.

Bug report: I noticed that subscriptions (at least from deviantart, I haven't confirmed it elsewhere yet) that don't finish their first run, (say it hits the 200 cap) on next runs, additional tags don't get imported and added to the image. This is important because it stops adding the artist tag!

Additionally, I cannot confirm if this happens if you leave the client open until the next run. I think every time this has happened. (5-6 times) I've closed the client and opened it the next day, and that's when it happens.

>>8265 Did you click the gear icon up and to the right of the "file import options" button and check the option that says "get tags even if url is known and file is already in db"? I had lots of trouble with this kind of thing before finding that

I haven't tried that, I enabled it "by default" and we'll see if that fixes it, thanks!

Not sure if this intended behavior or not, but I have some missing files in my hydrus, as you can see by the pic. Neither checking the database or a quick file integrity check seems to notice them, however.

>>8272 Thanks for the tip, but that does not appear to have resolved the issue!

Am I using the pixiv downloader wrong? I put in an artist number (I used 18127121), but it only downloaded 3 images. I did put account info in, and it tests OK. It looks like it downloaded all the single images the artist uploaded, but none of the albums. The detailed import status box says that they had "uninteresting mimes".

Three bugs, two annoying ones, one not a big deal: Annoying bug: if I reload a session that was using threadwatcher on certain boards on 4chan (all of the ones I rip from, which are /s/, /gif/, and /hr/) they will continue ripping the thread but will NOT include the filename. The only way to fix this behavior is to close that threadwatcher tab and create a new one with the same URL and tag options. less of a big deal bug, but the previous error makes this error a lot more annoying: I have a secondary monitor daisychained out of my primary monitor. When I plug it in, my computer will refresh the driver to accommodate the new display and resolution. Upon doing that, though, it causes Hydrus' pop-up menus to disappear entirely. I'm sure they're somewhere, but I can't find them with Alt+Tab or even Windows+Tab so it's easier to Esc out and restart Hydrus when this happens. Unrelated annoying bug: If you "collect" by a certain parameter, and then delete a bunch of files that are in that "collection", the thumbnail collection will show up on every highlight of other collections. I can repeat this behavior but not consistently. More specifically, if I "collect by pixiv_id" and then open a new tab on the pixiv_id I want to prune or delete, delete it, then go back to that search window with the "collect by pixiv_id", the pixiv_id tied to those deleted files will remain in the list until I open a new search window. Thank you very much for the software, it's a joy to use.

If you set 0.0 as a possible media zoom you'll get this error whenever you try to view something with it:ZeroDivisionError
float division by zero
Traceback (most recent call last):
File "include\HydrusPubSub.py", line 135, in Process
callable( *args, **kwargs )
File "include\ClientGUICanvas.py", line 2218, in ProcessApplicationCommand
self._ProcessApplicationCommand( command )
File "include\ClientGUICanvas.py", line 4356, in _ProcessApplicationCommand
command_processed = CanvasMediaList._ProcessApplicationCommand( self, command )
File "include\ClientGUICanvas.py", line 1752, in _ProcessApplicationCommand
self._ZoomIn()
File "include\ClientGUICanvas.py", line 1971, in _ZoomIn
self._TryToChangeZoom( new_zoom )
File "include\ClientGUICanvas.py", line 1879, in _TryToChangeZoom
zoom_ratio = new_zoom / self._current_zoom
ZeroDivisionError: float division by zero >>8284 The pixiv downloader doesn't download any mangas or ugoiras. You'll have to use a different program like pixivutil2 or wait for a newer downloader with manga/ugoira support.

>>8260 I don't know what I am doing wrong then This is how it looks for me You can see here that the artist does exist: https://www.pixiv.net/member.php?id=5681795 Yet it doesn't find any urls

>>8260 >If it is something like C:\users\yõu\desktop\someprogramyouonceextractedbutneverusedagain, then we may be in luck. Yeah, it was some dumb shit I don't even remember installing. I uninstalled it and removed it from the PATH and that fixed it. Thanks a lot.

>>8266 >>8265 >>8272 >>8273 >>8282 Thank you for this report. This has been fixed in the v297 release I will put out later today. Please let me know if you have any further problems. The fix will apply to future DA files. I move that 'gear icon' option to tag import options as well this week. You may like to use it to retroactively get tags for the files it missed previously. (run a manual DA download page for your sub with that option turned on, and it'll get the tags for those files) >>8275 Thank you for this report. What was the 'file domain' for that search page? The file domain is the search space as listed on lefthand button on the floating autocomplete dropdown. Is it 'my files' there, or 'all local files', or 'all known files'? If it is 'all known files', that includes files not in your client (like files you have deleted) but still tagged in the righthand 'tag domain'. This would be normal. But if it is 'my files' and you are seeing thumbnailless/remote files like this, then we do have a problem. >>8288 Thank you for these reports. The threadwatcher session tag loss bug is the same thing that was affecting Deviant Art and should also be fixed in today's v297. It will be fixed for all future parses. Let me know if you have any further problems with it. When you say 'pop-up menus', do you mean the messages that pop-up in the bottom-right of the main gui window? If you are a Windows user with floating tag input autocomplete dropdown lists, do these also break when this happens? What about if you have a 'review services' window (which should also not dissimilarly 'float' on top of the main gui) open? What happens if you jiggle or resize the main ui around (forcing a move event that may rescue the popups if they are offscreen)? You can force some popups to appear, btw, by going help->debug->make some popups. I will look into the collection delete issue, thanks. I expect the 'refresh your internal tags' calculation isn't being propagated up the right way. >>8289 Thank you–I have made sure to clear out <= 0.0 values when that option is saved. >>8290 Sorry–I misread your problem as a tag problem originally. I am also afraid that this worked ok on my machine. I got 14 files. If you go to network->manage pixiv account and click 'test', does it go through ok? I'm also wondering if the client thinks it is logged in but actually isn't, so maybe it is parsing a login page or something for you. The new login manager, which pixiv now works on, is still very barebones. There aren't a lot of debug or other 'hey, reset my login cookie' buttons yet. Please hang in there–I'll see if I can do something here for next week. >>8295 Great, thanks for letting me know!

>>8298 Great to hear on on all fronts. >When you say 'pop-up menus', do you mean the messages that pop-up in the bottom-right of the main gui window? No, I mean, for example, "Edit file import options" or "Edit tag import options" or, if I am remembering correctly, "similar files maintenance processing"

>>8298 >If you go to network->manage pixiv account and click 'test', does it go through ok? Yes >I'm also wondering if the client thinks it is logged in but actually isn't, so maybe it is parsing a login page or something for you. I don't know how to see that >Please hang in there–I'll see if I can do something here for next week. Ok thanks

>>8300 today:
2018/03/08 14:06:12: hydrus client started
2018/03/08 14:06:22: booting controller...
2018/03/08 14:06:23: booting db...
2018/03/08 14:06:25: updating db to v297
2018/03/08 14:06:26: updated db to v297
2018/03/08 14:06:27: preparing disk cache
2018/03/08 14:06:32: preparing db caches
2018/03/08 14:06:50: booting db...
2018/03/08 14:06:50: initialising managers
[xcb] Unknown request in queue while dequeuing
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python2: xcb_io.c:165: dequeue_pending_request: Warunek zapewnienia `!xcb_xlib_unknown_req_in_deq' nie został spełniony.


>>8298 Sure enough, the subscriptions forgetting the author and other settings does appear to be fixed, thanks!

Tried to see how the 4chan parser works to see if I can contribute in some way, got an exeption when a link for test data goes to a 404: NotFoundException
Traceback (most recent call last):
File "include\ClientGUIParsing.py", line 2653, in wait_and_do_it
network_job.WaitUntilDone()
File "include\ClientNetworking.py", line 1702, in WaitUntilDone
raise self._error_exception
NotFoundException
This is in manage parsers -> 4chan thread api -> edit -> fetch test data from url. on 404 you get an exeption, I feel like this is so much under construction you'll probably catch this one yourselve but still. Keep up the good work my man!

>>8318 also on test parse for that bad data (
fetch failed:

) the window for the parsed data gets filled with this exeption:
Exception:
ParseException: Unable to parse that JSON: No JSON object could be decoded
Traceback (most recent call last):
File "include\ClientGUIParsing.py", line 4550, in TestParse
results_text = obj.ParsePretty( example_parsing_context, example_data )
File "include\ClientParsing.py", line 1585, in ParsePretty
all_parse_results = self.Parse( parsing_context, page_data )
File "include\ClientParsing.py", line 1562, in Parse
posts = formula.Parse( parsing_context, converted_page_data )
File "include\ClientParsing.py", line 417, in Parse
raw_texts = self._ParseRawContents( parsing_context, data )
File "include\ClientParsing.py", line 1141, in _ParseRawContents
raise HydrusExceptions.ParseException( 'Unable to parse that JSON: ' + HydrusData.ToUnicode( e ) )
ParseException: Unable to parse that JSON: No JSON object could be decoded
same applies, you'll probably catch this yourself but still.

>>8298 >Thank you for this report. What was the 'file domain' for that search page? The file domain is the search space as listed on lefthand button on the floating autocomplete dropdown. Is it 'my files' there, or 'all local files', or 'all known files'? >If it is 'all known files', that includes files not in your client (like files you have deleted) but still tagged in the righthand 'tag domain'. This would be normal. >But if it is 'my files' and you are seeing thumbnailless/remote files like this, then we do have a problem. I'm pretty sure it was under all files, but it seems to have fixed itself by now. My guess is that since I have a fuckhuge hydrus and deleted a lot of files (those files were probably from when I had ~18k files and deleted around 1.5k) it was already in the todo list of the DB, but it still hadn't gotten to clearing those files, so when I asked for it to check, it didn't find any new pics, there were only those that it had already set aside to delete later. I accidentally left my hydrus on for a couple days in a row while seeding torrents, so it probably got the time to clear those pics out during this time.

I'm >>8094 and this is still happening in v297. Sometimes it crashes after I click the export files button, but before the window is opened.

Not sure how much of a bug this is because it could just be something that was overlooked, but when processing duplicates and choosing "this is better" the source URLs are not transferred to the new image from the old like the tags are. I don't use ratings enough to tell if they transfer.

Hey dev When doing a search for "similar to this picture", a lot of the results that turn up are… missing or I deleted them and hydrus forgot I deleted them. All the database checks, file integrity, etc, report no issues.

>>8329 > when processing duplicates and choosing "this is better" the source URLs are not transferred to the new image from the old Depending on if the source url is the url to the less good image or not this seems like a good idea. if it's actually a link to the aggrgate page of that artist then it should be copied I think.

Sorry for the late replies here lads, this week is killing me. >>8300 >>8310 Can you tell me which flavour of Linux you are running and which window manager you use? Most of these kinds of 'can't initialise gui' problems are due to a GTK incompatibility of some sort or another. I build on Ubuntu 16.04, so if your flavour is way out of step of that, that could be another reason. You might want to check out my running_from_source guide, which a dll-related suggestion you can try with my build, or if you have the python experience, you can even actually try to run it right from source: http://hydrusnetwork.github.io/hydrus/help/running_from_source.html If you try to run client_debug (if you aren't already), do you get any more interesting output to console? That typically prints boot loader debug info, but it might give a bit more here. >>8301 Thanks–that is interesting. I am not sure what is going on, but I guess the window reinitialisation gives a moment of like 'the current resolution is 0x0, and so the menus are being scaled down for an instant and then not back up again afterwards). Does the layout of the window change, to hide these? Can you take a screenshot of what it looks like afterwards, or is the original thing to click on still there but the menu just doesn't appear when you click on it? If you hit help->debug->force a gui layout now, does that help anything? >>8307 I wanted to write some ui code to help display the current login cookies for this week, but I have like two hours left and don't think I'll be able to fit it in. I'll see about adding a 'clear login data for pixiv/hentai foundry' menu entry instead that will help us figure out what is going on here. If pixiv hasn't fixed itself for you by the time you get v298, please try hitting that new entry, wherever I put it, and then try getting something from pixiv again. >>8318 >>8319 Thanks–is this on the default entry https://a.4cdn.org/tg/thread/57806016.json ? Unfortunately, the API call times out and 404s along with the main thread. You can try it in your browser as well and you should get the same 404. Please try an updated one–just go to 4chan, find a thread: https://boards.4chan.org/k/thread/37136033 And then convert it to the JSON API URL manually: https://a.4cdn.org/k/thread/37136033.json That should work! Note it is good to have a good-looking URL in the 'example urls' area because hydrus will use the URLs here to match up the parser with valid URL Classes as we share all this stuff with other users. >>8322 Ok. If it happens again or you see remote/deleted files in a different unexpected context, please let me know. >>8323 Thank you. Do you mean is can crash as you click thumbnail right-click menu->share->export->files? Can you describe what you have done in the ~60 mins leading up to that event? Maybe importing files by dropping them onto the client? Or using the manage tags dialog? If you boot the client fresh and do nothing but try to export some files, can you still get a crash that way?

[Expand Post]

Approx how many files are we talking, btw? Is it 1-100, or like 10k+? (Most of the crashes I have dealt with are due to certain objects not being deleted at the right time. Events like 'open a new window' can cause an internal garbage collector to finally delete some outstanding stuff, and it is only then the crash occurs. It may be that the crash-event for you is occuring a while before the export files dialog, but that that is causing the garbage collect event.) >>8329 >>8343 Yeah, I am not totally sure what to do here. Whether a URL should represent a fixed file or a dupe isn't a simple problem in a world where Cloudflare will non-reliably 'optimise' file content for the same URL and the boorus will swap out files for higher quality versions. I recently split up URLs into different URL Classes in the new domain manager, so I suspect I will revisit this at some point and permit some URLs to be merged across dupes but not others. All this info is still in your db, so I will see if I can figure out a retroactive fix when we get around to it. >>8337 Thank you for this report, this is useful. Have you done a bunch of duplicate file filtering? I believe that at some point, this was not deleting files in quite the right way, leaving these orphans. I specifically wrote a new db routine recently to try to deal with this, so since we have a good test case here, I would like to know if I have fixed it properly. Please get one of these pages with the bad results open. What happens if you change the file domain from 'my files' to 'all known files'? Do they disappear? Anyway, get it back so it shows bad results. Then hit database->maintain->clear orphan file records. Once that is done, refresh the page–is it fixed? If not, what if you then hit database->regen->similar files search tree?

>>8343 >>8348 Most of my "this is better" duplicates are optimized versions of the worse file. I usually download them from whatever website though Hydrus, and it gives the downloaded image a source URL as where it downloaded it from. Then I export, optimize, import, and use the dupes filter to choose the one with the smaller filesize. In this case the URLs should be transferred. Maybe it could be an option to transfer the URL or not. Speaking of URLs, it would be really nice if the source URLs listed on the boorus were picked up by Hydrus as well.

>>8348 >>8355 >Maybe it could be an option to transfer the URL or not. in the duplicates filter you can, in the top middle with the cog icon, change the data copied when you set the relationships. I think you can set this there. pic related

>>8348 >Thanks–that is interesting. I am not sure what is going on, but I guess the window reinitialisation gives a moment of like 'the current resolution is 0x0, and so the menus are being scaled down for an instant and then not back up again afterwards). Does the layout of the window change, to hide these? Can you take a screenshot of what it looks like afterwards, or is the original thing to click on still there but the menu just doesn't appear when you click on it? > >If you hit help->debug->force a gui layout now, does that help anything? I could take a screenshot but it would be a typical Hydrus window, just faded out because the unavailable window (in a test I just did, it was Review Services) had GUI focus but, obviously, wasn't there. Debug Force GUI doesn't help, no. Just messing around before hitting submit, and I was able to move the window around and I was able to snap this. I was able to replicate it by turning off then on my DisplayPort monitor while running Hydrus, so before doing it the windows were fine and afterwards were not.

>>8356 I can't find anything for URLs, only tags and ratings.

I keep getting this error while doing a tumblr download & deleting unwanted incoming files
error
C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:4045: error: (-215) dsize.area() > 0 || (inv_scale_x > 0 && inv_scale_y > 0) in function cv::resize

Traceback (most recent call last):
File "include\ClientCaches.py", line 2063, in DAEMONWaterfall
self.GetThumbnail( media ) # to load it
File "include\ClientCaches.py", line 1966, in GetThumbnail
hydrus_bitmap = self._GetResizedHydrusBitmapFromHardDrive( display_media )
File "include\ClientCaches.py", line 1792, in _GetResizedHydrusBitmapFromHardDrive
path = self._controller.client_files_manager.GetResizedThumbnailPath( hash, mime )
File "include\ClientCaches.py", line 1087, in GetResizedThumbnailPath
self._GenerateResizedThumbnail( hash, mime )
File "include\ClientCaches.py", line 341, in _GenerateResizedThumbnail
thumbnail_resized = HydrusFileHandling.GenerateThumbnailFromStaticImage( full_size_path, thumbnail_dimensions, fullsize_thumbnail_mime )
File "include\ClientImageHandling.py", line 258, in GenerateThumbnailFromStaticImageCV
thumbnail_numpy_image = EfficientlyThumbnailNumpyImage( numpy_image, dimensions )
File "include\ClientImageHandling.py", line 54, in EfficientlyThumbnailNumpyImage
return cv2.resize( numpy_image, ( target_x, target_y ), interpolation = cv2.INTER_AREA )
error: C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:4045: error: (-215) dsize.area() > 0 || (inv_scale_x > 0 && inv_scale_y > 0) in function cv::resize


>>8348 I have extensively used the similar file filter, yes. >Please get one of these pages with the bad results open. What happens if you change the file domain from 'my files' to 'all known files'? Do they disappear? Nope, they're still there. >Anyway, get it back so it shows bad results. Then hit database->maintain->clear orphan file records. Once that is done, refresh the page–is it fixed? If not, what if you then hit database->regen->similar files search tree? There are no orphan file records to clear. They did clear only after "regen > similar files search tree". Thanks, and hope this helps!

>>8389 Symbol such as ' ♡ ' Appear as 3 line when par of a sentence/title. such as in this example: みんなで作ってア・ラ・モード♡ Is there a setting a touched inadvertendly for this, or it this part of the pixiv remodel project?

This might be intended behavior, but number of tags doesn't ignore blacklisted tags. It counts all tags, local or not, causing searches like pic related.

>>8392 This has to do with windows and the fonts it uses. the heart is double booked with the 3 line symbol. I think I got foobar to display the heart properly by changing the font to segoe ui or something

>>8356 >>8355 >>8381 Yeah, you can't merge URLs yet–URLs were not a hydrus-recognised 'datatype' when I wrote the dupe filter. I will have to revisit it when I am on better footing with what URLs go where. I will add support for adding URLs with the new gallery parsers as I roll out the new download engine. It should be totally doable to recognise a source URL on a gallery page and associate it additionally as you download the file. >>8363 Thanks for the screenshot. That is pretty weird! I assume this is some trickery in your driver somewhere that isn't happy reinitialising 'stay on parent' windows like review services. Some force-relayout call isn't being propagated down. I have written a new entry at help->debug->force a layout for all tlws now for today's release. Please give it a go sometime and let me know if it changes this issue at all. It will print a bunch of related size/pos info to the screen as well–a screenshot of that (or copy/paste from the logfile, which will have the same info) would be helpful. >>8382 Thank you for this report. Can you name the tumblr that does this? Does it have any unusual images at all, either in ratio or size? It looks like somehow I am asking CV to resize to negative coordinates, which maybe suggests one of these images is reporting a bad size. I will write a neater catch for this for today's release. Have you ever seen this error in a different context, or only on an actively importing tumblr downloader? >>8389 Great, thank you. I will write some extra orphan clearing for the search tree stuff as well. >>8392 >>8409 Thank you for this report. Unfortunately, I think I am misunderstanding it. That tag renders ok for me in the client (see my image, where I spammed it on a test file). Can you take a screenshot of what you see so I can better understand what is wrong here? I am guessing this is a font issue as >>8392 says. I am on Win10 English language with default fonts (which I guess is Segoe UI). >>8402 Thanks. This is not intended, but it is too CPU-intensive to fix at the moment (I would have to load the actual tag text for all files in the domain and then check them all off against the current tag blacklist before counting them up, which would be minutes of work). I expect in the next year or two to write a new database layer that will do the heavy lifting of calculating 'tags as we want them' to for siblings and tag cenorship and so on and storing them in a more permanent, revisitable way. When that is in, I can start generating better autocomplete numbers and doing numtag searches like this more accurately.

>>8382 >>8411 >Thank you for this report. Can you name the tumblr that does this? Does it have any unusual images at all, either in ratio or size? It looks like somehow I am asking CV to resize to negative coordinates, which maybe suggests one of these images is reporting a bad size. I will write a neater catch for this for today's release. It was ligbi.tumblr.com but none of the images seemed unusual. It had about 3,800 files so I doubt you want to sit through a download of it yourself. >Have you ever seen this error in a different context, or only on an actively importing tumblr downloader? Never seen it before, but I haven't used the tumblr downloader in quite some time. Really though, I was whipping through images and deleting them via the media viewer quite rapidly as new ones were incoming. I only got the error about 5-6 different times during that download and I believe it switched focus off the media viewer when it popped up.

>>8348 >Thank you. Do you mean is can crash as you click thumbnail right-click menu->share-export-files? yes >If you boot the client fresh and do nothing but try to export some files, can you still get a crash that way? also yes >Approx how many files are we talking, btw? Is it 1-100, or like 10k+? this happens even when I export a single file

>>8411 Summary of what I did, which is something I do basically daily. Additionally, this behavior is repeatable if my monitor is simply being woken up. -Unplugged displayport from work laptop. -Plugged it into the displayport-out of my primary monitor. -Windows refreshed the driver with the expected black screen. -Went to Hydrus, checked if I would see no popups. -Confirmed it, went to debug, and got this:

>>8413 Thanks. Yeah, that tumblr looks fine, unless there are some weird files as you say like 2k images deep. I am guessing this was some weirdness about the specific workflow. I bet it was something like you were processing files in the media viewer that hadn't generated thumbnails yet (because they were off-screen of the original thumbnail page), and some 'this file got deleted' event went through, causing the thumb to be generated off-screen, which something somewhere wasn't happy with. Do you by any chance have files set to remove from view when they are trashed? Anyway, I will keep this in mind. Please let me know if you see it again. >>8421 Thanks. I will give this another pass this week. >>8422 Thanks. It looks like the layout didn't do it, so I will write another debug action for v300 that'll be more rigourous. Do you remember which 'popups' you tried, specifically? Were any of them open before the drivers refreshed?

>>8436 >which popups Oh gosh I can't remember which one I used for this test, I think it was one of the dialogues for a threadwatcher. I can repeat it across any of them and then notate them tomorrow. >Were any of them open? No, these dialogues were closed or unopened beforehand. I guess due to how I noticed this bug I'm not actively doing something so there's no window activity.

>>8436 >Do you by any chance have files set to remove from view when they are trashed? No.

Subscriptions completely stopped working for me: I somehow managed to open two instances of the "manage subscription" window at once. So, I selected "check queries now" and applied the changes in the first window and then simply clicked on "cancel" on the second one. Since then none of my subscriptions have been able to check for new images including newly created ones. All of them are "working" and are next "checking on dialog ok" and when I deselect "check now" it changes into "imminent", but nothing happens in either case. I can't find anything unusual in any of the log files either.

Hydrus.Network.299.-.Linux.-.Executable arch user with multiple woes here, The first time I started the program it complained in terminal about missing /usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders.cache and told me to do the following, which I did as root: gdk-pixbuf-query-loaders > /usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders.cache It still worked fine, but I don't like error messages. All themes are broken and only when I use "GTK2_RC_FILES=/usr/share/themes/Raleigh/gtk-2.0/gtkrc" will I get a usable interface and no complaining like down below. Here are some logs (reduced because of repeated lines: (client:4303): Gtk-WARNING **: Error loading theme icon 'gtk-cancel' for stock: Unable to load image-loading module: /usr/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.so: /usr/lib/librsvg-2.so.2: undefined symbol: cairo_tag_begin

** (client:4303): WARNING **: Pixbuf theme: Cannot load pixmap file /usr/share/themes/Arc/gtk-2.0/assets/null.png: Unable to load image-loading module: /usr/lib/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so: /PATH/TO/hydrus network/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/lib/libpng16.so.16)

** (client:4195): WARNING **: Invalid borders specified for theme pixmap:
/usr/share/themes/Arc/gtk-2.0/assets/entry-active-notebook.png,
borders don't fit within the image

This line got spammed by the thousands:
(client:4195): GdkPixbuf-CRITICAL **: gdk_pixbuf_get_width: assertion 'GDK_IS_PIXBUF (pixbuf)' failed As far as I know all non-standard gtk2 themes use gtk-engine-murrine to draw their things, so perhaps including that would work. It would be cool if you could use gtk3 but I guess that is almost impossible considering how different it is from gtk2, wxgtk pheonix seems to have gtk3 support though. I can't press URL links, example when I try to click on the link from "help>about": (client:1843): GLib-GIO-WARNING **: /etc/xdg/kde-mimeapps.list contains a [Added Associations] group, but it is not permitted here. Only the non-desktop-specific mimeapps.list file may add or remove associations.
XPCOMGlueLoad error for file /usr/lib/firefox/libmozgtk.so:
/usr/lib/libgdk-3.so.0: undefined symbol: g_log_structured
Couldn't load XPCOM. I also can't open images in external applications because of this: kde-open5: /PATH/TO/hydrus network/libz.so.1: version `ZLIB_1.2.9' not found (required by /usr/lib/libpng16.so.16) It seems like the package doesn't use the included libpng library. I tried using "LD_PRELOAD=. ./client" but that didn't help either and had the same error messages as above. The mouseover bar from the duplicate viewer sometimes spazzes out and doesn't show again, forcing me to close it with an external program. I will try to take a look at the log next time it happens. Thanks for your work, I know that most of my woes will stop when I compile it myself but it seriously takes too long to compile wxgtk-phoenix.

Seems the thread downloader doesn't work for older threads for some reason. >>>/wx/35263 just gives me 404s, even though I can access the pics just fine. Other old threads like >>>/wx/33678 don't work as well, so I'm guessing it's because of the date. I scrapped /v/'s webm thread a while ago, so I can say for sure it's not because of the size. I tried it on v298 and it didn't work. I tried updating to v300 but it still won't download.