[DoctorD]

I got a lot done on Sunday. Here are all new features & changes:

1. Scrapes Data18 Web Content now. There's a new button that you must click when scraping this type of content called "Scrape Data18 WebContent".
2. Allows you to pick fanart when scraping JAV movies instead of just grabbing the jacket.
3. New preference to name nfo file movie.nfo instead of putting the movie name in the nfo file name. This option really only works well if your files are in their own folders first because otherwise movie.nfo will be overwritten when you scrape the next movie.
4. Better quality image scaling & improvements to the fanart picking GUI.
5. A bunch of other misc bug fixes. Complete list here: Jav Movie Scraper GitHub Commits Page

[AiWaBR]

Hello AiWaBR,

Are you using the newest build? I had fixed some issues with WebContent this morning. In general, webcontent scraping is a bit of a mess because data18 has 4 or 5 different page formats they use (that I know of!). If you or anyone else have a specific file that is not working or is missing data and you KNOW it has a page on data18.com, please reply to this thread with the URL on data18.com that is not scraping correctly.

Thanks DoctorD, downloaded the update and now is working very well.

Some scenes do not make the scrapper due to the site to change the title (eg bangbros) and not get the data18, ai gets old name.

The program for me that always wanted a program like this or a website to make scrapper scene is getting perfect, some changes can make it better, and I believe that's what you want too do.

Some suggestions and requests:

1 When ordering your poster image that is not the fanart. Or both, in order poster and fanart later.
Place 2 more option for editing and the scrapper is made as Studio, Parents, Genres, Parental rating.
Option 3 manually edit actors, add that does not come. (As often happens with those who did not have thumb on the data18).
4 One way to look at the scene by id data18.
Ex: http://www.data18.com/content/1129580 - the id of the scene is 112958
If you do not find the name of the scene, we can put the id that took the data18.
5 Edit the title to seek power, sometimes the main site (bangbros and other), change the title and can not find the old name.

I think that's what got me to remember while using the program. If you can add would show.

But thanks again for the great work, both in the xbmc scrapper as for this wonderful program.

[Chuck Bartowski]

Hi DoctorD - nice App

However i think something on the Data18 website may have changed again. It was previously picking up poster and fanart URL's however on certain Kink sites no longer seems to eg http://www.data18.com/content/1133528 where it picks up the "plot" and "actor" thumbs but for both poster and fanart just returns the "kink" logos.
Does seem to be kink specific eg http://www.data18.com/content/1134905

The webscarper within xbmc itself does pick up the poster and thumb - indeed for those two when you select "choose art" it brings up the whole selection - it is just fanart that is crippled but understand this is an xbmc issue rather than scraper

Really wish i could help you with this project but unfortunately my IT skills in terms of coding is zilch but very happy to help test

[DoctorD]

Hello Chuck Bartowski,

That issue should be fixed now. Just grab the latest build linked from the github page. Your testing is appreciated - data18 has a quite a few different page layout types and it's hard to get them all working (and stay working!) sometimes, especially when they go and change things all the time!

[Chuck Bartowski]

Thanks DoctorD -

At the moment the scraper seems to populate the nfo file with all posters and all fanart even where you only select 1

Did wonder whether any of the other media managers (eg ember Tiny etc) could read the nfo files and download the art but doesnt seem to work -

We really do need our own repository like TMDB :-)

[AiWaBR]

Thanks DoctorD -

At the moment the scraper seems to populate the nfo file with all posters and all fanart even where you only select 1

Did wonder whether any of the other media managers (eg ember Tiny etc) could read the nfo files and download the art but doesnt seem to work -

We really do need our own repository like TMDB :-)

I agree with you.

If I had a site like TMDB for adult content, I'm sure it would help a lot of people, myself help putting much content.

Unfortunately I know nothing about building a website, if it were more like TMDB, would help a lot.

[Pr.Sinister]

Please be patient guys as i am trying to get a website up for adult web content... it is harder than i expected and i want to have as much done as possible and then do tweaks later. Right now i am stuck at when you add an episode, you have to choose the season every time...

[DoctorD]

Hi Chuck,

Are you just trying to get all the the other posters / art to have them for your own review later? If you want to do that, enable downloading of extrathumbs in the preferences menu. You will need to have the file already in its own folder before scraping (and then select the folder, not the file) to use this option reliably. With this option checked, all posters/fanart are saved into extrathumbs. Then you can always rename them to something else or do whatever you want with them.

The reason I put all the posters in the nfo file is so that if you later decide you want a different one in XBMC, you can pick a different one within XBMC. This should work 100% time for the posters, but the fanart will have that issue with the not having a spoof attribute, so it might not work all the time for the fanart.

I can't say much about what you can do with other programs - I haven't used them in a while (except for Media Companion, which I rather like to browse the files after they are scraped). I was originally using ember media manager, but they took out support for custom XML scrapers in their beta version 1.4 which is why I wrote this standalone scraper program originally!

At one point, this project actually started out as a XML JAV scraper. It actually kind of worked, but the actor names were always a little off because at the time I was going solely by dmm.co.jp's thumbnail file names. Anyways, I probably won't release the XML version because then I would have spend time keeping it updated and it never worked all that well anyways.

However, if you or anyone else find another program that is helpful to use in conjunction with this one (or even one that is better), feel free to let us all know so we can all benefit. For example, has anyone else tried out this scraper https://github.com/laoyang945/javscraper? It's an XBMC XML JAV scraper I found through googling. I don't think it will scrape things in English, however, but it does appear to support some sites and content I currently don't...

I'll try to get to everyone's feature requests mentioned in this thread as I have time, but I mostly only work on this program a few hours per week, so it could take a while.

Happy scraping everyone and thanks for your feedback!

[Chuck Bartowski]

Thanks DoctorD. For some reason although the preference for downloading is set the programme doesn't seem to actually download the image files for the brazzers and kink sites (all I have tried so far)only write the nfo file. I will try again to double check .

Will also try the XML scraper.

I was also hoping for the rewrite of the XML scraper in Ember which was supposed to be coming

[DoctorD]

Is "Write fanart and poster files" checked in the preferences menu?

[Chuck Bartowski]

Hi DoctorD - yes it is checked and i scrape based on folder rather than file. The "in-progress" wheel keeps turning once the poster and fanart is selected and doesnt stop - however you can still select "write to file" which stops the process with an nfo file only but no artwork

[Chuck Bartowski]

Please be patient guys as i am trying to get a website up for adult web content... it is harder than i expected and i want to have as much done as possible and then do tweaks later. Right now i am stuck at when you add an episode, you have to choose the season every time...

That's great news and we wait patiently and appreciate your efforts

[AiWaBR]

Please be patient guys as i am trying to get a website up for adult web content... it is harder than i expected and i want to have as much done as possible and then do tweaks later. Right now i am stuck at when you add an episode, you have to choose the season every time...

Great news, we'll be waiting patiently.

Thanks for your work.

[laodong0112]

Hi DoctorD,
Thanks for your great work.Can you add an option to swith off google tanslate,personally I'd rather using japanese metadata than using english!Thank again.

[salinus]

Hi,

I made a fork of the scrapper with some optimization and refactoring, accessible through the fork button on github.
-manually add genres and actors
-double click on the film choser
-some new scrapers
-using actors from iafd instead of data18
-rename files
-refactoring code mostly for the gui, for easier use

I use a different file layout, which can be seen in the renamer settings, so its possible the scraper does not work for you.
There are probably new bugs and defect features, so its more for someone who wants to look into the code.