Control Web Crawler Jobs

POST

https://api.devrev.ai/web-crawler-jobs.control

POST

/web-crawler-jobs.control

1 curl -X POST https://api.devrev.ai/web-crawler-jobs.control \
2      -H "Authorization: Bearer <token>" \
3      -H "Content-Type: application/json" \
4      -d '{
5   "action": "abort",
6   "id": "foo"
7 }'

Try it

200webCrawlerJobWebCrawlerJobsControlExample

1 {
2   "web_crawler_job": {
3     "id": "foo",
4     "accept_regexs": [
5       "foo"
6     ],
7     "created_by": {
8       "display_id": "foo",
9       "id": "foo",
10       "display_name": "foo",
11       "display_picture": {
12         "display_id": "foo",
13         "id": "foo",
14         "file": {
15           "type": "foo",
16           "name": "foo",
17           "size": 42
18         }
19       },
20       "email": "foo",
21       "full_name": "foo",
22       "state": "active"
23     },
24     "created_date": "2023-01-01T12:00:00.000Z",
25     "description": "foo",
26     "display_id": "foo",
27     "domain_names": [
28       "foo"
29     ],
30     "frequency": 42,
31     "max_depth": 42,
32     "modified_by": {
33       "display_id": "foo",
34       "id": "foo",
35       "display_name": "foo",
36       "display_picture": {
37         "display_id": "foo",
38         "id": "foo",
39         "file": {
40           "type": "foo",
41           "name": "foo",
42           "size": 42
43         }
44       },
45       "email": "foo",
46       "full_name": "foo",
47       "state": "active"
48     },
49     "modified_date": "2023-01-01T12:00:00.000Z",
50     "no_parent": true,
51     "notify_on_complete": true,
52     "num_bytes": 42,
53     "num_timeout_urls": 42,
54     "num_urls_scraped": 42,
55     "reject_regexs": [
56       "foo"
57     ],
58     "sitemap_index_urls": [
59       "foo"
60     ],
61     "sitemap_urls": [
62       "foo"
63     ],
64     "state": "aborted",
65     "urls": [
66       "foo"
67     ],
68     "user_agent": "foo"
69   }
70 }

Change state of a web crawler job to pause or resume it back to running.

Request

This endpoint expects an object.

actionenumRequired

Allowed values:

idstringRequiredformat: "id"

The ID of the job.

Response

The response to resume a web crawler job.

web_crawler_jobobject

Control Web Crawler Jobs

Headers

Request

Response

Errors