/hydrus/ - Hydrus Network

Archive for bug reports, feature requests, and other discussion for the hydrus network.

Index Catalog Archive Bottom Refresh
Name
Options
Subject
Message

Max message length: 12000

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)
Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

Uncommon Time Winter Stream

Interboard /christmas/ Event has Begun!
Come celebrate Christmas with us here

8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.
(12.82 KB 480x360 0LQlfkvQ3Ok.jpg)

Version 343 hydrus_dev 03/13/2019 (Wed) 21:49:44 Id: 44b5d1 No. 11863
https://www.youtube.com/watch?v=0LQlfkvQ3Ok windows zip: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.Windows.-.Extract.only.zip exe: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.Windows.-.Installer.exe os x app: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.OS.X.-.App.dmg linux tar.gz: https://github.com/hydrusnetwork/hydrus/releases/download/v343/Hydrus.Network.343.-.Linux.-.Executable.tar.gz source tar.gz: https://github.com/hydrusnetwork/hydrus/archive/v343.tar.gz I had an ok week. The Client API v1.0 is essentially finished, and I did some code cleanup as well. client api The Client API now has full 'file search' capability, including fetching file metadata like size and resolution and tags, and also file and thumbnail fetch. As before, the help for these new commands is here: https://hydrusnetwork.github.io/hydrus/help/client_api.html But for some tiny last cleanup jobs, the Client API v1.0 is now complete. From now on, I will fit adding additional commands to the API in normal weekly 'small job' work. I am overall pleased with how this has gone. If you have been waiting for a 'working' version of the API to play with, now is the time. For regular users, there is a link to a browser add-on in that help file, and it seems a phone app is also in the works. the rest The media viewer is now smarter about recycling its internal windows, so browsing between video and static image types (in either direction) is now faster and smoother. If you often get the one-frame 'tiny grey square' when transitioning between video and static images, this should now be eliminated. More types of tiff are now supported, and I fixed an issue that was mis-detecting soundless webms as mkvs. The list of file paths in the manual file import window is now sortable, and it will preserve this sort order in the subsequent file import or path tagging dialog (useful if you wish to set up sequential number tagging, or you would like for some reason to import by largest file first). It also has a new '#' column to let you return to the original parse sort order. full list - client api: - fixed an int/str type mismatch issue with service_names_to_actions_to_tags in /add_tags/add_tags in the client api that meant that argument was not working - fixed up some last /get_files/search_files stuff - added /get_files/file_metadata - added /get_files/file - added /get_files/thumbnail - added help and unit tests to reflect the above - updated client api version to 4 - . - the rest: - the list of paths in the manual file import dialog is now sortable. this order will be preserved in regular and 'add tags' ok events for this dialog. it has a new '#' column so you can return to 'parse' order if desired - animation and static image windows in the media viewer canvas are now recycled through media type transitions, making for slightly smoother browsing between mixed media - increased aggression of media viewer image prefetch - added support for 'MM' Tiffs - fixed webm mime parsing for webms with no audio (these were falling back to mkv)
[Expand Post]- improved the error reports when a serialised png fails to import - the mass-open-urls popup is now pausable as well as cancellable - fixed several recently broken ui unit tests - misc old code cleanup - some misc test controller/constant refactoring next week I'll finish up the last Client API work and move on to tag OR searching in the client as the new 'big job'. My hope is this will not be a gigantic project, but we'll see how it shakes out. Beyond that, I'd like to fit in some small jobs–maybe some time units on the animation scanbar, maybe some new shortcuts. This week got away from me a bit, so I also have a ton of bug reports and other messages to catch up on.
Anonymous 03/13/2019 (Wed) 22:03:45 Id: 6b29d5 No. 11864
>>11863 Thanks Dev
Anonymous 03/13/2019 (Wed) 23:18:54 Id: 6b29d5 No. 11865
(132.10 KB 1440x862 Untitled.jpg)

>>11863 the import files window reset its dimensions; i can manually resize the left/right sides but the top/bottom ones can't be resized. Settings the values in options didn't work either. Unless that only takes effect after a restart…?
hydrus_dev on new tablet not set up 03/14/2019 (Thu) 00:23:14 Id: 44b5d1 No. 11867
>>11865 Thanks, the new list here is too tall. I will reduce its min height for 344.
hydrus_dev Board Owner 03/14/2019 (Thu) 00:32:53 Id: 44b5d1 No. 11868
>>11867 Test I now have a hydrus tablet, hoping I can keep up with messages better like this.
Anonymous 03/14/2019 (Thu) 15:05:46 Id: 4881c6 No. 11873
>>11863 Get an error using API /get_files/file_metadata
The repository encountered an error it could not handle! Here is a dump of what happened, which will also be written to your client.log file. If it persists, please forward it to hydrus.admin@gmail.com:

Traceback (most recent call last):
File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/home/kyck/.local/lib/python3.6/site-packages/twisted/_threads/_threadworker.py", line 46, in work
task()
File "/home/kyck/.local/lib/python3.6/site-packages/twisted/_threads/_team.py", line 190, in doWork
task()
--- <exception caught here> ---
File "/home/kyck/.local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
result = inContext.theWork()
File "/home/kyck/.local/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
File "/home/kyck/.local/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/home/kyck/.local/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
return func(*args,**kw)
File "/home/kyck/hydrus/include/ClientLocalServerResources.py", line 1243, in _threadDoGETJob
media_results = HG.client_controller.Read( 'media_results_from_ids', file_ids )
File "/home/kyck/hydrus/include/HydrusController.py", line 605, in Read
return self._Read( action, *args, **kwargs )
File "/home/kyck/hydrus/include/HydrusController.py", line 192, in _Read
result = self.db.Read( action, *args, **kwargs )
File "/home/kyck/hydrus/include/HydrusDB.py", line 955, in Read
return job.GetResult()
File "/home/kyck/hydrus/include/HydrusData.py", line 1519, in GetResult
raise e
include.HydrusExceptions.DBException: TypeError: 'int' object is not iterable
Database Traceback (most recent call last):
File "/home/kyck/hydrus/include/HydrusDB.py", line 581, in _ProcessJob
result = self._Read( action, *args, **kwargs )
File "/home/kyck/hydrus/include/ClientDB.py", line 9259, in _Read
elif action == 'media_results_from_ids': result = self._GetMediaResults( *args, **kwargs )
File "/home/kyck/hydrus/include/ClientDB.py", line 6053, in _GetMediaResults
( cached_media_results, missing_hash_ids ) = self._weakref_media_result_cache.GetMediaResultsAndMissing( hash_ids )
File "/home/kyck/hydrus/include/ClientCaches.py", line 1913, in GetMediaResultsAndMissing
for hash_id in hash_ids:
TypeError: 'int' object is not iterable
hydrus_dev Board Owner 03/14/2019 (Thu) 17:06:24 Id: 44b5d1 No. 11874
>>11873 Thank you for this report. I will fix this for next week. This has revealed a flaw in my testing regime that I will make a longer term job to fix as well.
Anonymous 03/14/2019 (Thu) 21:10:20 Id: 53d6c0 No. 11876
>>11863 I can't access the repo. If I do what is asking, same error but with a dialog window instead of a toast. 2019/03/14 22:03:16: Exception: 2019/03/14 22:03:16: CancelledException Upload cancelled: This hydrus service (hydrus service: official) recently failed to log in. Please hit its 'refresh account' under 'review services' and try again. Traceback (most recent call last): File "include\HydrusThreading.py", line 342, in run callable( *args, **kwargs ) File "include\ClientGUI.py", line 4134, in _THREADUploadPending service.Request( HC.POST, 'update', { 'client_to_server_update' : client_to_server_update } ) File "include\ClientServices.py", line 879, in Request network_job.WaitUntilDone() File "include\ClientNetworkingJobs.py", line 1155, in WaitUntilDone raise HydrusExceptions.CancelledException( message ) include.HydrusExceptions.CancelledException: Upload cancelled: This hydrus service (hydrus service: official) recently failed to log in. Please hit its 'refresh account' under 'review services' and try again.
hydrus_dev Board Owner 03/15/2019 (Fri) 00:51:22 Id: 44b5d1 No. 11878
>>11876 Thank you for this report. If you check the service's review services page, what status does it report there? Normally, right under its address it says 'service is functional' and under the account section it says 'account is functional'. Do you have a specific error there with more info? This error is something I have been trying to pin down a while. It seems to fix itself after a few days, maybe when some maxed bandwidth rules clear out. Does that panel have any odd bandwidth stuff going on?
Anonymous 03/15/2019 (Fri) 03:03:41 Id: 53d6c0 No. 11879
(33.59 KB 644x265 hydrus.jpg)

>>11878 "Unknown account" but no issue with bandwidth or any such thing.
hydrus_dev Board Owner 03/16/2019 (Sat) 20:56:25 Id: 44b5d1 No. 11909
>>11879 Ok, thank you. I will keep working on this. If you unpause the service on that panel, does the 'Repository is paused!' change to an error, or does it think it is fine?
Anonymous 03/16/2019 (Sat) 21:32:23 Id: 53d6c0 No. 11910
>>11909 I think I did it with the same error when I first tried, however now it allowed me to unpause and refresh account worked. I guess as you said "It seems to fix itself after a few days" XD
Anonymous 03/18/2019 (Mon) 12:31:05 Id: 7e8b61 No. 11928
>>11639 here's the read out I get from the subprocess report mode when I try to open a webm, its the same as when I try to import pngs. I'm also posting what I get when I do manage upnp, though I'm not sure if this is relevant to me. I'm on 343 now.
Anonymous 03/18/2019 (Mon) 12:47:30 Id: 03fbe2 No. 11929
(127.83 KB 801x1280 hydrus_mobile.jpg)

Hey dev, mobilefag here. Got a quick demo up and running. I've put in the ability to manage accounts (in case you have several different hydrus clients/access keys) and a browse page which just grabs some files from the /search_files endpoint for the time being. Glad to see the progress being put into the client api! I'm going to have to do some lurking when it comes to understanding the importing/scraping features seeing as I don't use them at all. However I figure it's important for other users and it's a part of the api so I want to include it as well. One thing I noticed when toying around with the client api was searching using namespaces seems to be wonky. -namespace:* does not work in client api -namespace: does not work in client api namespace:* does not work in client api namespace: DOES work in client api As you can see, negating does not work at all, and non-negating searches only work without the star (even though in the program itself either are fine). And are "system:" namespaces available for this endpoint? I tried "system:rating for Quality = rated" (A numerical rating service) for example and got nothing Anyway some features I'd really love : - A tag search endpoint! I feel this is a must for me before I'd consider release. I need to be able to query for what tags are available, so that I can power an autocomplete search in the app. That and be able to list/manage tags in their own page. - Pagination support for search endpoints. If I could send an items_per_page, and page parameter to fetch items chunks at a time that'd be great - Sort support. The app has a great sort feature that's currently absent from the api. it would be nice to be able to sort by image size, random, etc.
Anonymous 03/18/2019 (Mon) 21:14:17 Id: 6b29d5 No. 11932
>>11929 How does this work? Do you leave Hydrus running in the background and so long as the computer is awake you can access it via mobile? I don't understand. t. brainlet
hydrus_dev Board Owner 03/18/2019 (Mon) 22:19:39 Id: 44b5d1 No. 11933
>>11928 Thank you. If you open a terminal and type 'which ffmpeg', where does it think it is? The important value in your env there is the 'PATH'. It looks like /usr/bin and /usr/local/bin are in there, which are the usual missing suspects here. Is your ffmpeg somewhere else? Also, if you go to install_dir/db/client - 2019-3.log and search for 'TERM_PROGRAM' to get to where this section was written to log, it looks like there should be a little bit more than what fit in the popup window. I do not think it is important, but is there LD_LIBRARY_PATH or LD_LIBRARY_PATH_ORIG in there, and if so, what is the value (the text after the the name and colon)? Please do check and scrub out anything personal again, although I think these values should be innocuous. If your ffmpeg got installed to an unusual location not in the PATH there, then if you make a softlink to it like so: ln -s /path/to/ffmpeg/ffmpeg /usr/local/bin/ffmpeg Does it then work?
hydrus_dev Board Owner 03/18/2019 (Mon) 23:13:57 Id: 44b5d1 No. 11935
>>11929 Hey, this looks fantastic! Yeah, I only have basic tag support in atm. Wildcard searches are actually dealt with in hydrus with a different search object, so what is being searched for there is literally the asterisk character rather than 'anything'. Namespace searches are the same–they use different code to run faster. It is unusual that 'namespace:' would work as a fixed tag but the same negation wouldn't, but this is an edge case anyway that isn't expected to ever run. I expect to add namespace and wildcard searches to the Client API in my future weekly work. I will be adding more system search predicates as well. Supporting tag autocomplete search sounds good as another extension. Perhaps until then you could have a 'favourites' list or similar to make quick-entry easier? Unfortunately, hydrus does not support paginated searches like a booru would. It returns everything in one go, without sorting. If you have a technical limit to the number of thumbs you can display, I recommend you break the results list into chunks of your own preferred page size. Sorting results is unfortunately not (yet) possible at the db level, so you will have to do it yourself using data from the file_metadata calls. Please keep me updated on how you get on here. >>11932 I can't speak confidently for exactly how his system works, but yeah, you would have to have the client on for this to work, and maybe set up NAT port forwarding or use hydrus's in-built UPnP port forwarding, and then point your account settings in his phone app at either your fixed IP address and port or get yourself a no-ip.org kind of deal, like I do for the PTR.
Anonymous 03/18/2019 (Mon) 23:42:38 Id: 03fbe2 No. 11936
>>11935 Thanks for the answers. Pagination is not a huge deal because thankfully most endpoints simply return ids or file hashes which I can then use to query files in chunks myself. And yeah like you said, I could passively store tag information as I fetch files and keep a list. That should work ok until we get a tag search endpoint. >>11932 As long as the hydrus app is running and has the client api enabled then yes, you can access it via the mobile app. You just need to set up access via review services > client api > add > from api request. You do that and enter your computers IP address and the port the api is using into the app and you're good to go. This won't work outside your LAN though. You might need to take some extra steps to allow public access to your server. Steps like port forwarding, setting a static ip (for convenience's sake so you don't have to update your ip whenever it changes) and maybe even a domain name if you want to be extra fancy. This isn't unique to hydrus or my mobile app though, these are the kinds of steps you would need to take to open up any kind of server to the world wide web that you're running on your home computer. If you're running a web server or ftp server or something it's the same deal.
Anonymous 03/18/2019 (Mon) 23:47:12 Id: 53d6c0 No. 11937
>>11936 Out of curiosity, what platforms are you ultimately planning to release it for?
Anonymous 03/19/2019 (Tue) 00:37:34 Id: 7e8b61 No. 11938
>>11933 I did which ffmpeg and got nothing so I brew installed it and now everything works great. No idea why I didn't have it. Thanks alot for the help
Anonymous 03/19/2019 (Tue) 01:16:55 Id: 18ab7c No. 11939
>>11937 I am developing using flutter which is cross platform. That means it will be available for iOS and Android. Submitting to iOS app store can be a headache though so I wouldn't be surprised if Android would be released first. However my intention is to release it for both
Anonymous 03/19/2019 (Tue) 02:15:45 Id: 9c564b No. 11940
>>11939 If you plan to make it FOSS please submit it to the F-Droid.
Anonymous 03/19/2019 (Tue) 03:44:40 Id: 6b29d5 No. 11942
>>11936 I see. So I can shitpost my memes from mobile yet leave my collection in Hydrus and not waste my limited phone space. I can use this. >>11940 Yes please this. I'd rather F-droid than google play anyway, which I use for Clover as is.
Anonymous 03/19/2019 (Tue) 06:01:41 Id: 18ab7c No. 11945
>>11940 >>11942 I hadn't heard of F-Droid before. I'm not familiar with what needs to be done to release there. I'll definitely look into it and try my best to get it submitted. Hopefully it's as simple as submitting the same apk you'd submit to the play store.
hydrus_dev Board Owner 03/19/2019 (Tue) 19:34:24 Id: 44b5d1 No. 11947
>>11929 >>11932 >>11937 >>11940 >>11942 I should say at this point that the Client API is still prototype, so atm it is http only, which means comms are unencrypted. This is fine when on the same machine or your LAN, but for remote connections, this is not great. Before we jump on doing a phone app for real-world use, I will have to add https (albeit with self-signed certificates, since we are talking IP addresses and not paid-for domains).
hydrus_dev Board Owner 03/19/2019 (Tue) 20:04:22 Id: 44b5d1 No. 11950
>>11938 Great, I am glad we sorted it out!
hydrus_dev Board Owner 03/19/2019 (Tue) 20:05:43 Id: 44b5d1 No. 11951
>>11938 >>11950 I should say, I'll improve the error reporting here.
Anonymous 03/19/2019 (Tue) 21:48:58 Id: 6b29d5 No. 11954
>>11947 Eh, it'll get there eventually. I still haven't imported most things since I want to preserve original creation date. Although I guess the chan filenames being unix time means they could be converted later
Anonymous 03/19/2019 (Tue) 22:05:02 Id: 03fbe2 No. 11955
(97.36 KB 640x641 f7215076-af0f-442c-a17b-539a429c277f.jpg)

>>11947 I agree. at the moment im only testing on lan. I have no problem waiting for https support before releasing. By the way I had a quick question on the file metada endpoint. The example on the site shows the following bit of json for tags : "service_names_to_statuses_to_tags" : { "local tags" : { "0" : [ "blonde hair", "blue eyes", "looking at viewer" ] "1" : [ "bodysuit" ] } } I was wondering, why are the local tags split up into two separate arrays? Why is bodysuit separate from the other tags? Why not have a single array like so : "service_names_to_statuses_to_tags" : { "local tags" : [ "blonde hair", "blue eyes", "looking at viewer", "bodysuit" ] } Should I expect any number of arrays or is it always 2?
Anonymous 03/19/2019 (Tue) 22:19:25 Id: 53d6c0 No. 11956
>>11947 Yeah, encryption would be pretty important. I personally don't care because lol openvpn server at home, but that's not normal.
hydrus_dev Board Owner 03/20/2019 (Wed) 18:27:57 Id: 44b5d1 No. 11964
>>11955 The numbers there are the 'status' of that list of tags. In hydrus, a file's tags can have four different statuses: 0 - current - the file has the tag for this service 1 - pending - this tag is pending to the service 2 - deleted - this tag was current but has now been deleted 3 - petitioned - this tag is current and is petitioned for deletion Pending/petitioned relates to the 'pending' menu in the main hydrus client gui. Repository tags are not fixed as current/deleted until pending/petitioned are commited through that menu. In writing this out, I realise I fucked up here because a 'local tags' service won't ever have 'pending' tags. I will rewrite this help section to have it refer to a hypothetical tag repository.
hydrus_dev Board Owner 03/20/2019 (Wed) 18:35:39 Id: 44b5d1 No. 11965
>>11964 To follow up here, if you wish to edit a file's tags with /add_tags/add_tags, you will need to use this more detailed tag status information to figure out if you are 'petitioning' or 'rescind pending' or whatever. For practical purposes, if you want to calculate what tags the file currently 'has' for easy phone display, you can go: tags = ( current ∪ pending ) - petitioned Petitioned tags are pretty rare, so you can probably just go ( current ∪ pending ). You'll then be unioning those sets across all given service_names to get a file's 'all known tags' as you would see in the client.
Anonymous 03/20/2019 (Wed) 20:36:09 Id: 18ab7c No. 11971
>>11964 >>11965 Ah. I forgot tags can have statuses. Thanks for clarifying
Index Catalog Archive Top Reply
1
Forms
Delete
Report
Quick Reply