/hydrus/ - Hydrus Network

Archive for bug reports, feature requests, and other discussion for the hydrus network.

Index Catalog Archive Bottom Refresh
Name
Options
Subject
Message

Max message length: 12000

files

Max file size: 32.00 MB

Total max file size: 50.00 MB

Max files: 5

Supported file types: GIF, JPG, PNG, WebM, OGG, and more

E-mail
Password

(used to delete files and posts)

Misc

Remember to follow the Rules

The backup domains are located at 8chan.se and 8chan.cc. TOR access can be found here, or you can access the TOR portal from the clearnet at Redchannit 3.0.

Uncommon Time Winter Stream

Interboard /christmas/ Event has Begun!
Come celebrate Christmas with us here


8chan.moe is a hobby project with no affiliation whatsoever to the administration of any other "8chan" site, past or present.

(104.98 KB 1366x768 neat.png)

Anonymous 12/08/2015 (Tue) 10:41:42 Id: 528e0b No. 1553
Reposting from >>>/tech/457184 Maybe relevant to your interest. Japan did it. Automatic anime images tagging with neural networks. THE FUTURE IS NOW http://illustration2vec.net/ http://illustration2vec.net/papers/illustration2vec-main.pdf
Would be awesome to see this implemented. Some kind of ability to retrain with new data sets, if at all feasible, would greatly add to the value too. The current net is pretty solid for anime stuff, but if you're trying to work with a different kind of image set it's only so so. And even with anime, the data used to train the current system was presumably a lot smaller than it could have been, due to being a proof of concept and all.
Since this post, hydrus has had some better video support, maybe it's worth referencing other automatic taggers here. This one takes a video file and creates a list of tags+timecodes, it was initially built to create supercuts of video files containing just the parts that contain the tags you specify. I think it can be used with images as well. https://github.com/ryanjay0/miles-deep Yahoo made one for images that just judges what is and isn't NSFW, doesn't tag them any further though. https://github.com/yahoo/open_nsfw Warning: nightmare fuel https://open_nsfw.gitlab.io/
(121.65 KB 734x660 h1.png)

(41.36 KB 576x512 h2.jpg)

From >>>/v/12285254 https://paintschainer.preferred.tech/ It would be interesting to one day have a "colourise" option in image-viewers in the same way we have things like rotate, scale, filters, etc. Would be very neat for black and white manga.
>>6129 That Html-table-parser-python3 that you're using crashing when attempting to be installed, when looking for a "setup.py" in the location it's looking for.
>>6130 fixed this, there is problem due to that package is not yet on pypi. so now it have to be cloned first and install it from source.
>>6133 Attempting to install the dependancy "Html_Table_Parser_Python3" the way you've got it written down (pip install git+git://github.com/schmijos/html-table-parser-python3.git ) causes pip to throw an error with setup.py not being found. What you actually want is (pip install git+git://github.com/rachmadaniHaryono/html-table-parser-python3.git ) works fine. Running the code as-written from there throws this error message: Traceback (most recent call last): File "/usr/local/bin/i2vec-cli", line 11, in <module> load_entry_point('i2vec-cli==0.1.0', 'console_scripts', 'i2vec-cli')() File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 561, in load_entry_point return get_distribution(dist).load_entry_point(group, name) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 2631, in load_entry_point return ep.load() File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 2291, in load return self.resolve() File "/usr/local/lib/python2.7/dist-packages/pkg_resources/init.py", line 2297, in resolve module = import(self.module_name, fromlist=['name'], level=0) File "/usr/local/lib/python2.7/dist-packages/i2vec_cli/main.py", line 15, in <module> from html_table_parser import HTMLTableParser File "/usr/local/lib/python2.7/dist-packages/html_table_parser/init.py", line 1, in <module> from .parser import HTMLTableParser File "/usr/local/lib/python2.7/dist-packages/html_table_parser/parser.py", line 13, in <module> from html.parser import HTMLParser ImportError: No module named html.parser
>>6137 You may want to use python3, because the program only support python 3 version
>>6137 You disgust me Okay, so after rerunning everything in python 3.5 this time, reinstalling those dependancies, and getting Geckodriver to start, i finally got it running as intended. And…
User@Anon's computer ~ $ i2vec-cli '/path/to/hydrus network/db/client_files/f80/80bf66b390fbb1554ff556a1e878cdd2acffe42a05f83dafbb5cf9695e1c9a11.jpg'
path:80bf66b390fbb1554ff556a1e878cdd2acffe42a05f83dafbb5cf9695e1c9a11.jpg
[[['#', 'General Tag', 'Confidence', '']],
[['#', 'Character Tag', '', '']],
[['#', 'Copyright Tag', '', '']],
[['#', 'Rating', '', '']]]
Running the same image with –format Hydrus nets you
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.1.0', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 83, in main
print(convert_raw_to_hydrus(tags))
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 63, in convert_raw_to_hydrus
result.append('rating:{}'.format(dict_result[key][0]))
IndexError: list index out of range
>>6149 thanks for the bug report, now v0.1.1 should fix that. can you check if there is any error?
>>6155 No programming errors. Still, the program doesn't actually give out any sort of tags. Testing it with pic related on my own gives out a whole bunch of valuable tags (Even correctly identifying it as a 2hu, which I honestly wasn't expecting.) whereas the command line tool gives the following result
User@Computer ~ $ i2vec-cli '/path/to/file/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg' --format hydrus
path:54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg
2017-06-15 03:53.28 key v=Rating
Good on you for responding to everything, by the way.
>>6157 Forgot my pic
>>6158 unfortunately i can't reproduce it https://github.com/rachmadaniHaryono/i2vec_cli/commit/94b422330776d8710b0ee686eda4727743295a67 but with that version, it may help me debug it. the program now support url and i can get the tag from that url directly
i2vec-cli -nc https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg --format hydrus
url:https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg
1girl
blonde hair
solo
yellow eyes
long hair
breasts
smile
open mouth
arms up
armpits
cleavage
arm up
:d
looking at viewer
star
v
from above
foreshortening
pointing
series:touhou
rating:safe
>>6162
User@Computer ~ $ i2vec-cli https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg/54d5bb6151a4263ee67e904041%E2%80%A6.jpg --format hydrus
url:https://media.8ch.net/file_store/54d5bb6151a4263ee67e904041366bf6c52ed46b0006ef604da1f77184c3c0d3.jpg/54d5bb6151a4263ee67e904041%E2%80%A6.jpg

User@Computer ~ $
Still gets me nothing, even after upgrading Requests. I can see the firefox window open up, and the file being uploaded, but the window closes far too fast to get any of the tags away from the window. I'd suggest creating a small delay between finishing to upload everything and getting the tags, since that's where it seems to stumble upon.
>>6165 https://github.com/rachmadaniHaryono/i2vec_cli/commit/68f76b484cca5e4603b6ce6fd36ebefc3e7a1d66 now with close delay i'm really interested about this error and want to fix it for the problem you have
>>1554 If you add something like this to the program, I have to ask you if you can also make a 'user to approve tag' section, and if an image is tagged exclusively with tags from the algorithm, a mandatory 'tag manually' tag This would just keep potential crap tags out and also denote that the image may have more tags that are obvious but were not tagged. — it's only after I wrote this that I seen how old the op was, but point stands, would rather have something like this in a quarantine to confirm section rather than seamless
I tried installing and I got the same problem as >>6137 only I don't know how to do it with python 3. Can you tell me how?
>>6173 use
pip3 install -e .
to make it explicitly install with python3. you may want to use `–user` if you don't want to sudo the command. also you may have to uninstall the previous installation
pip uninstall .
i will put it in doc
>>6174 Thanks! I managed to install it. I also needed to install geckodriver btw. Now I try it with the 2hu of >>6158 and I get (same if I do >>6162 >>6165):
$ i2vec-cli 2hu.jpg
path:2hu.jpg
[[['#', 'General Tag', 'Confidence', '']],
[['#', 'Character Tag', '', '']],
[['#', 'Copyright Tag', '', '']],
[['#', 'Rating', '', '']]]
Also not sure if it's supposed to be like that but it opened a firefox window with http://demo.illustration2vec.net/ and then it closed that window again Also it saves a geckodriver.log every time I use it
>>6175 >Thanks! I managed to install it. I also needed to install geckodriver btw. i will implement option to use other driver such as phantom js or chrome or maybe another feature to remove geckodriver log as well >Also not sure if it's supposed to be like that but it opened a firefox window with http://demo.illustration2vec.net/ and then it closed that window again yes becaues firefox is the default driver for the program for this picture i'm still looking for it. there is 'close delay' feature which will pause the program before the program end, and user can examine if anything is wrong with the upload.
>>6170 Thanks for the close-delay, that really helps with debugging this stuff out. Here's what i got so far.
$ i2vec-cli '/path/to/2hu/image/Weapon accuracy.jpg' --close-delay 60
path:Weapon accuracy.jpg
[[['#', 'General Tag', 'Confidence', '']],
[['#', 'Character Tag', '', '']],
[['#', 'Copyright Tag', '', '']],
[['#', 'Rating', '', '']]]
I took a screencap of what was going on with the open Firefox window so you could see what's going on there. As you can see, it wasn't what i thought, that the tags just weren't being displayed,but it's just that grabbing the table and displaying the contents in a command line doesn't happen.
>>6180 that is quite unexpected. if the table is not recognized, the 4 header row (general tag, character tag, etc) should also not recognized now i add `dump html` option to dump the table to html file. if it is correct it should write html and when that file opened in browser it should be like screenshot
>>6187 The option, if anybody is checking, is –dump-html.
>>6278 thank you for the example, i will implement it
http://illustration2vec.net/papers/illustration2vec-main.pdf >We browse several web services (e.g., Danbooru and Safebooru) and collect 1,287,596 illustrations and the associated metadata https://nyaa.pantsu.cat/view/887969 >http://gelbooru.com/ ID:1 to ID:3470395 Seems like the model could be improved if someone has the storage and compute power. Then there are 1.7 million more from zerochan. There are slso some site rips on rutracker, for example a partial one of sankakucomplex. https://rutracker.org/forum/viewtopic.php?t=3641643
Okay, so i finally managed to make the dump-html thing work. The secret was to not to wait at the > /usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py(61)get_tags()
-> dump_html_to_file(bb_tag.html)
(Pdb)
prompt like i was expecting, but to type in "BB_tag.html". Here's the Html prompt that was spat out. This was copied and pasted from my terminal, but it still reads as a valid web page when copied and pasted into html. >The body was too long. Dammit, codemonkey. https://pastebin.com/RC0RfLdb
>>6180 >>6295 the update have been uploaded. it is now using only requests libray and parsing only json response from the server. it is using method mention by >>6278
>>6316
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.1.1', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.5/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 143, in main
session = Session(driver=driver)
TypeError: __init__() got an unexpected keyword argument 'driver'
>>6332 >>6193 hi, i just made server feature for this program. this also fix typeerror on init method.
>>6333 Cool trips.
Traceback (most recent call last):
File "/usr/local/bin/i2vec-cli", line 11, in <module>
load_entry_point('i2vec-cli==0.2.0', 'console_scripts', 'i2vec-cli')()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 561, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2649, in load_entry_point
return ep.load()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2303, in load
return self.resolve()
File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2309, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
File "/usr/local/lib/python3.5/dist-packages/i2vec_cli/__main__.py", line 20, in <module>
import peewee
ImportError: No module named 'peewee'
(367.85 KB 1360x638 i2vec-cli - Chromium_033.png)

>>6426 peewee already on requirements.txt. but on next version all package required will be on setup instead of requirements there is ongoing index page work in progress
>>6428 running pip install peewee seems to have fixed the "tags not being read from the page" problem. Consider all of these bug reports fixed. >>6332 >>6295 >>6180 >>6175 >>6165 >>6157 >>6149 >>6137 >>6130 I'm gonna start using this a lot now, thanks.
Necrobumping for great justice
>3 years ago Oh fuck.


Forms
Delete
Report
Quick Reply