Gregory's Server@bangskij It should allow you to paste a URL you want to be crawled. (Careful with the depth setting! Something like 12 or 18 will crawl the whole internet...) Also, when I tried it, there was a proxy function. Set your browser to use the YaCy proxy and it will crawl all sites you visit.
|
2 comments
@datenritter I set it to crawl google at 7 levels and it kept stopping for no memory, but it did spike into 50mbits. Restarting it now with more RAM allocated. Off to sleepy, but this is very exciting. Can't believe I can have my own search engine. |
@datenritter I managed to open the port, connect properly to the network and get a crawl starting. It's using very little bandwidth, about 1-2 megabits with the odd spike into 10. Considering I have a thousand of those and use them for email and this social network I think I should be good. It still posts an error message but it is also crawling so, guess it's fine. I don't know how to proxy my browser through it though but it's late here, try again tomorrow.