>>11275 Thank you for this report. Please turn off all idle processing under options->maintenance and processing–does that stop the freezes? Is your db on an HDD or a SSD?

>>11318 I've been running from source for a couple of days now and it hasn't frozen since.

when manually fixed "bytes > str" error, anoter one appear: happens where symbols like ☆ are in neughbour txt files' tags UnicodeDecodeError 'charmap' codec can't decode byte 0x98 in position 35: character maps to <undefined> File "C:\Users\alexander\.virtualenvs\Hydrus_source-nKsCVeBo\lib\site-packages\wx\core.py", line 3259, in <lambda> lambda event: event.callable(*event.args, **event.kw) ) File "C:\Hydrus_source\include\ClientThreading.py", line 382, in wx_code self.Work() File "C:\Hydrus_source\include\HydrusThreading.py", line 659, in Work self._work_callable() File "C:\Hydrus_source\include\HydrusData.py", line 1306, in call self._func( *self._args, **self._kwargs ) File "C:\Hydrus_source\include\ClientGUIImport.py", line 1625, in RefreshFileList self._paths_list.UpdateDatas() File "C:\Hydrus_source\include\ClientGUIListCtrl.py", line 1042, in UpdateDatas ( display_tuple, sort_tuple ) = self._GetDisplayAndSortTuples( data ) File "C:\Hydrus_source\include\ClientGUIListCtrl.py", line 591, in _GetDisplayAndSortTuples ( display_tuple, sort_tuple ) = self._data_to_tuples_func( data ) File "C:\Hydrus_source\include\ClientGUIImport.py", line 1579, in _ConvertDataToListCtrlTuples tags = self._GetTags( index, path ) File "C:\Hydrus_source\include\ClientGUIImport.py", line 1596, in _GetTags tags = filename_tagging_options.GetTags( self._service_key, path ) File "C:\Hydrus_source\include\ClientImportOptions.py", line 368, in GetTags txt_tags_string = f.read() File "C:\Users\alexander\.virtualenvs\Hydrus_source-nKsCVeBo\lib\encodings\cp1251.py", line 23, in decode return codecs.charmap_decode(input,self.errors,decoding_table)[0]

>>11321 Thank you, this should also be fixed for v336. I missed some file i/o encoding, thinking it would default to utf-8. If you want to fix it yourself in the source, find the various open( path, 'r' ) calls across the program and insert an encoding param, like so: with open( path, 'r', encoding = 'utf-8' ) as f: Only the 'r' and 'w' calls–the 'rb' and 'wb' ones are raw bytes and have no encoding to do.

>>11211 I wanted to see if it would improve, and general usage seemed to be better but had another huge memory explosion on Hydrus shutdown, so py3 did not help that lock up. >100% disk usage thrashing again >14GB RAM

On Hydrus 336 it looks like auto updating the PTR is causing Hydrus to lock up, anyway I can disable updates for it?

Trying to switch between regular mode and dark mode causes an error 01:31:29 AM: Debug: ClientToScreen cannot work when toplevel window is not shown double free or corruption (out) Aborted (core dumped)

>>11357 >>11347 Thank you, I now get this on my admin client on shutdown–it just hangs for me with 2GB-ish (which is way more than normal) and almost zero CPU/HDD after the processing transaction completes. I will do a full pass of all my thread shutdown signalling this week. You can alter specific maintenance processing rules under options->maintenance and processing and pause all repo processing under the services->pause menu. You can pause specific repos (which includes processing) under services->review services->remote->the repo->pause button.

>>11374 Thank you for this report. I will check it out.

I am getting this error traceback in the log: 2019/01/19 16:25:40: Traceback (most recent call last): File "include\HydrusDB.py", line 743, in MainLoop self._InitDiskCache() File "include\ClientDB.py", line 6533, in _InitDiskCache new_options = self._GetJSONDump( HydrusSerialisable.SERIALISABLE_TYPE_CLIENT_OPTIONS ) File "include\ClientDB.py", line 5002, in _GetJSONDump return HydrusSerialisable.CreateFromSerialisableTuple( ( dump_type, version, serialisable_info ) ) File "include\HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple obj.InitialiseFromSerialisableInfo( version, serialisable_info ) File "include\HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo self._InitialiseFromSerialisableInfo( serialisable_info ) File "include\ClientData.py", line 1083, in _InitialiseFromSerialisableInfo loaded_dictionary = HydrusSerialisable.CreateFromSerialisableTuple( serialisable_info ) File "include\HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple obj.InitialiseFromSerialisableInfo( version, serialisable_info ) File "include\HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo self._InitialiseFromSerialisableInfo( serialisable_info ) File "include\HydrusSerialisable.py", line 268, in _InitialiseFromSerialisableInfo value = CreateFromSerialisableTuple( serialisable_value ) File "include\HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple obj.InitialiseFromSerialisableInfo( version, serialisable_info ) File "include\HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo self._InitialiseFromSerialisableInfo( serialisable_info ) File "include\HydrusSerialisable.py", line 268, in _InitialiseFromSerialisableInfo value = CreateFromSerialisableTuple( serialisable_value ) File "include\HydrusSerialisable.py", line 100, in CreateFromSerialisableTuple obj.InitialiseFromSerialisableInfo( version, serialisable_info ) File "include\HydrusSerialisable.py", line 156, in InitialiseFromSerialisableInfo self._InitialiseFromSerialisableInfo( serialisable_info ) File "include\ClientImporting.py", line 1396, in _InitialiseFromSerialisableInfo ( self._automatic_archive, self._exclude_deleted, self._min_size, self._min_resolution ) = serialisable_info ValueError: need more than 3 values to unpack Is there any way to fix this? Id prefer not to have to try and redo ~4 years of imports and tagging.

>>11399 Thank you for this report. We can fix this without losing all your shit, no worries, but it might take a little back and forth and you'll maybe have to reset your file->options settings. Can you say which version this is? It doesn't look like current code. And which version, if any, are you updating from? Do you have a backup of the pre-update db?

It seems that disabling a subscription popup (with sync progress) makes hydrus client wait for a subscription to complete (with downloading all files), and exiting client forcibly leads to to losing some data.

>>11420 >>wait for a subscription to complete i mean on client shutdown

>>11399 Thank you very much. As a side note, only the actual files and tags matter to me. If lose everything else, that is perfectly fine. Looking back it was an obviously terrible idea, but I was attempting to update from version 294 to version 334 before attempting the clean install. I remember it seemed to successfully update the db through a few versions before crashing.

Got some kind of bug that im not sure what difference it makes, unlike the memory explosions everything seems to be normal besides this popup. It only started showing up on v336 >System Limit:10 >Every time I roll over refresh, the program lags for a second then spits out the message But everything seems to be normal anyways? SystemError <class 'wx._core.MenuEvent'> returned a result with an error set

>>11421 >>11420 Thank you for this report. I have been doing a bunch of shutdown related cleanup this week and have now improved the 'should I stop working?' checks in the subscription system to respond better to client shutdown. It should now abandon after the current file/gallery page being worked on finishes. Please let me know if this continues to cause you trouble. It should be a clean shutdown, but if you get some spammy-but-harmless 'fugg ShutdownException' popups in the last moments of the client, please send them in and I'll quieten them.

>>11424 I think you probably hit some bitrot here. The update code of 334 is probably applying wrongly to something from v294. If you have a backup of v294, please roll back to it and try updating maybe 5 or ten versions at a time, so like do 299, 304, 309… If you do not have a backup, please go to your install_dir/db directory and run the sqlite3 executable. Copy/paste the following: .open client.db
SELECT version FROM version;
.exit What version does it say there? That's probably the version it 'succeeded' to. Let's say it was 303. Try then installing 304, which is more likely to do the 303->304 update step successfully than 336 can. It may still fail, in which case we'll have to figure out something cleverer.

>>11426 Thank you for this report. Can you check your "install_dir/db/client - 2019-01.log" file, scroll to the bottom, and find the full traceback for one of these errors? This was the right-click menu on the thumbnail area, right? So each entry was ok except 'refresh'? wx is a bit finicky with menus sometimes. I think there are internal limits on some menu-binding code, and when it fails due to a one-in-ten-thousand event of a cache running out of free ids, it then can't figure out how to highlight or show an icon or whatever when you mouse over. Was this problem just on the same menu, do you remember, or did it come back when you right-clicked again? Did you notice any other menus acting odd or not refreshing? Some of this stuff is due to me trying to make wx menu code do weird dynamic stuff, just pushing it a bit hard. I am not yet expert enough to figure out how I can detect and recover from these issues, so the general default solution here is to restart the program. Please let me know if you keep getting it and if you discover any sequence of events that make it more likely.

Sadly I do not have a backup handy. It recognizes it as version 325. Looks like it made it a bit farther than I thought. Im going to download the 325 version. Should I attempt to install it or do I need to do something else first?

>>11434 Just install as normal. It may give the exact same error, but if it breaks for any reason trying to update, a one-version attempt shouldn't alter anything. If you have an external usb drive or something to make a backup now, I would recommend it before you do try updating, just so we have a spare copy should anything new go wrong. If you haven't got the space for everything, see if you can squeeze the space to make a copy of all four client*.db files in install_dir/db–this is really what your db is and is what is being affected here by the updates.

>>11436 Alright, I just copied all the files from the db folder except the actual client files (does that count as a db backup?), and have attempted the install. It opened this time, and it appears client file folders f00 - f0b are missing. I don't see them anywhere, including the recycling bin (and I don't remember deleting them), so as far as I'm aware they're just gone. These folders appear to be a small subsection of the files though, so even if they're gone for good, that's still much better than losing everything. Should I just recreate folders f00 - f0b or is there something else I can / need to do?

>>11433 >This was the right-click menu on the thumbnail area, right? Right thus far it only effects the right click menu in the GUI, The general steps I took was >Setting a system limit >opening the internal preview browser >deleting some pictures one by one with right click instead of the fast archive/delete >closing the preview after getting through all 10 >right click to refresh (where the bug would show up) I did run across another instance of it today, but not in the same way. This time it was in the rightclick menu when scrolling over pic in the download gallery options, I was going to paste something but noticed the bug again. And on more experimenting it looks like the GUI option [Manage] when highlighting a file trips the bug. Also if I right click the blank canvas space, but chose nothing and close it it also trips the bug. I might try to get a video of this later. The tracebacks exceptions from the 2019-01 file is huge, but searching for the specific one only got a few duplicates and I dont think it will help nail anything down.
<class 'wx._core.MenuEvent'> returned a result with an error set


2019/01/23 12:13:58: Uncaught exception:
2019/01/23 12:13:58: SystemError

<class 'wx._core.MenuEvent'> returned a result with an error set


2019/01/23 12:14:02: Uncaught exception:
2019/01/23 12:14:02: SystemError

<class 'wx._core.MenuEvent'> returned a result with an error set


2019/01/23 12:14:12: Uncaught exception:
2019/01/23 12:14:12: SystemError

<class 'wx._core.MenuEvent'> returned a result with an error set


>>11441 Something to note, like the memory freeze up problems this error seems to go away completely upon restarting hydrus, when I first restarted hydrus yesterday I thought the problem fixed itself but after having the bug again today I think it could be related to idling or just the straw that broke hydrus's back after something broken being left alone for hours eventually jamming the wheels (in this case overnight). I wont be able to get a video until it happens again, but shouldnt take too long.

>>11437 >>11436 So I went ahead and created f00-f0b directories and got the thing open. Should I expect it to crash when a file is unable to load or will there be an error I can use to prompt me to search the tags at the given source location? Or will it just remove the (supposedly) now faulty db entries to the missing paths?

>>11437 >>11436 >>11443 Which .db file is are the tags and source stored in? (assuming you're able to say for whatever reason) It appears an error is thrown for every missing file, so if I can query for the identified filename in the sqlite program, I can theoretically get the corresponding tags and source location. Using those, I should be able to manually find each missing file again to replace them… unless the database automatically erases the entry after finding it missing. I may also be able to search for the missing locations and find the filenames and tags based on that if having the browser run into them automatically erases the entries. I had a class on mysql and the queries seem to be nearly identical, so I should supposedly be able to figure out the basic queries needed if I know where i'm looking.

>>11437 >>11436 >>11443 >>11444 Ok, Sorry to make so many useless posts in such a short amount of time, but I have been experimenting and found that the files do not get automatically removed or deleted. They stay in the view and just throw an error to the log. As this is the case, I can just wait and whenever I find a file that is missing, I will re-download it. That being said, it would be much easier if there was a way to sort by filename and / or to download the image without having to copy the url and pasting it to the url download page. Are there any tricks to do this? I would like to thank you very much for all your help. I will be sure not to make such a stupid mistake again.

>>11443 >>11437 >>11444 >>11446 You did right recreating those missing folders. If it helps, every file missing from those folder will start with the characters '00', '01'', … '09', '0a', '0b'. If you want to do it in SQLite, you might like to try something like: SELECT hash FROM current_files NATURAL JOIN hashes WHERE service_id = x AND hash LIKE ?%; I am sorry, I forget that 'LIKE' syntax exactly. You'd have to pass the '?' as a byte. That might be too much of a pain. I recommend you hit database->check->file integrity->quick and then tell it to export a .txt with URLs for the missing files. This will remove the file entries in your db, so if you still want to have a record of the missing somewhere for some SQL experimentation, make a backup somewhere. Otherwise, that routine will dump a huge list of URLs you can then feed back into hydrus's raw url downloader to recover. I recommend you read the 'help my db is broke.txt' under the install_dir/db directory, if you haven't already. That several big folders disappeared is pretty worrying. I would guess it was a serious hard drive fault. Anyway check out that document as background reading, just in case it helps. Anyway, let me know how you get on!

>>11442 >>11441 Thank you for this. I am afraid I cannot reproduce this. Strange that there is no good traceback here–it looks like a wx-level thing at the C/C++ level that isn't giving nice python line information. I just did some research, and this 'error set' issue may be an actual bug in wx. I assume >>11442 is still on 336? 337 should be less aggressive on memory and fixes the shutdown issue as well. If your machine has been having trouble with that related stuff, and this is a system level memory tracking/allocation issue, maybe this problem is relieved as a result. In any case, please keep me updated on this. I'll keep thinking about this, and–EDIT: I just checked and there's a new wx 4.0.4, here https://www.wxpython.org/ . I'll see if I can update to that this week, maybe it will magically fix in 338.

>>11459 >I'll assume you're still on 336? Nope, I'm on 337 the bug still shows up, but only after heavy use or idle not quite sure which, today I ran lots of downloaders from gelbooru but didnt open the preview window. Messing with it more it appears that a lot of the main program menu also trips the same wx. popup error, it's not all the options but it looks like whichever ones it is, is still consistent on breaking (such as rolling over refresh in the rightclick context menu). As promised, I got a screencapture of it. But the res is shit, my fault for not double checking the recording settings in OBS. Since the bug is gone on restart I wont be able to get a clearer picture of it today. https://streamable.com/h4il8

Trying to use sort by ratings on v338 gives me a ValueError traceback information ValueError not enough values to unpack (expected 3, got 2) File "include\ClientGUICommon.py", line 1495, in EventSortTypeChoice self._UpdateAscLabels( set_default_asc = True ) File "include\ClientGUICommon.py", line 1436, in _UpdateAscLabels ( asc_str, desc_str, default_asc ) = media_sort.GetSortAscStrings() I think it works, but I can't search for asc/desc results, only the default method of sorting

>>11510 I can confirm this bug on 388. Every single 'sort by' ratings, custom ones or default gave me the same bug readout. The ratings sort don't work at all, and there's a blank dialogbox where should be ascending/descending

>>11462 Thank you, this vid is very helpful. It looks like you have a pretty heavy client, with a lot of pages open and a ton of pending tags. Do you have, say, more than 100 pages open? I am wondering if your client is running out of ids for menus quicker because it has all the other stuff going on. Is it possible you can temporarily close some of those pages, restart the client, and see if that eases things up a bit? Have you tried 'page of pages', btw? This helps nest pages Hit F9 (i.e. choose a new page) and then special->page of pages. You can drag and drop page tabs onto that new page 'folder' and you won't be navigating left and right across a long ribbon of tabs.

>>11510 >>11517 Thank you for this report. I apologise, I fucked up a change here. I will have it fixed for 339.

Hi, having a small problem with the Manage Import Folders option.
TypeError
CallBlockingToWX() missing 1 required positional argument: 'func'
Traceback (most recent call last):
File "include\HydrusThreading.py", line 342, in run
callable( *args, **kwargs )
File "include\ClientGUI.py", line 2608, in THREAD_do_it
controller.CallBlockingToWX( wx_do_it )
TypeError: CallBlockingToWX() missing 1 required positional argument: 'func'
Not a problem with export folders.

>>11532 Thank you for this report. I made a couple of stupid mistakes last week–this was one. I am sorry for the problem. It will be fixed in tomorrow's release.

This thread is getting laggy, so: N E W E W T H R E A D H R E A D >>11542 >>11542 >>11542 >>11542 >>11542