How to batch crawl？

[zglaozhu]

Can this tool only be crawled individually?
May I ask how to batch crawl？

[enenumxela]

You can use --seeds to specify batch URLs. --domain must be specified with this.

[zglaozhu]

You can use --seeds to specify batch URLs. --domain must be specified with this.

Sorry, I'm still not sure，In the Windows10 system, I have a urls file 1. txt
like
http://t.github.com/1
http://t.github.com/4

What command should I use to batch scan?
Like xcrawl3r -- seeds -- domain 1. txt -- user agent -- debug - o 2. txt?

[enenumxela]

You can use --seeds to specify batch URLs. --domain must be specified with this.

Sorry, I'm still not sure，In the Windows10 system, I have a urls file 1. txt like http://t.github.com/1 http://t.github.com/4

What command should I use to batch scan? Like xcrawl3r -- seeds -- domain 1. txt -- user agent -- debug - o 2. txt?

xcrawl3r --seeds 1.txt --domain github.com -o 2.txt

Support URLs with the same root domain only

[zglaozhu]

Support URLs with the same root domain only

For different root domain？
like
http://t.github.com/1
http://t.github.com/4
http://t.google.com/1
http://t.googlw.com/4

Like
xcrawl3r --seeds 1.txt --domain github.com,google.com, googlw.com -o 2.txt?

[enenumxela]

Support URLs with the same root domain only

For different root domain？ like http://t.github.com/1 http://t.github.com/4 http://t.google.com/1 http://t.googlw.com/4

Like xcrawl3r --seeds 1.txt --domain github.com,google.com, googlw.com -o 2.txt?

That's not supported yet. Nice Idea though will try implement the same

[zglaozhu]

Support URLs with the same root domain only

For different root domain？ like http://t.github.com/1 http://t.github.com/4 http://t.google.com/1 http://t.googlw.com/4
Like xcrawl3r --seeds 1.txt --domain github.com,google.com, googlw.com -o 2.txt?

That's not supported yet. Nice Idea though will try implement the same

Okay, thank you all the same