Gateways

Http Gateway

class novelsave_sources.utils.gateways.BaseHttpGateway[source]

Base gateway interface that defines http communication

abstract property cookies: requests.cookies.RequestsCookieJar

Get current cookies being used in session

The setter for this property must also be implemented.

Returns: The cookies in the session
Return type: RequestsCookieJar

get(*args, **kwargs)[source]: Aliased method to send GET request using request() method

post(*args, **kwargs)[source]: Aliased method to send POST request using request() method

abstract request(method: str, url: str, headers: Optional[dict] = None, params: Optional[dict] = None, data: Optional[dict] = None, json: Optional[dict] = None) → requests.models.Response[source]

Send an http request to the specified url using the specified options

Parameters

method (str) – The method of request to send. ex: GET, POST, PUT
url (str) – The endpoint to which the request to be made
headers (dict) – The headers to be send with the request. If not specified sends default headers from requests module.
params (dict) – The query parameters to be send with the request.
data (dict) – ‘x-www-form-urlencoded’ to be send with the request.
json (dict) – json to be sent in the request body.

Returns

The response resulting from the request

Return type

requests.Response

Default http gateway

class novelsave_sources.utils.gateways.DefaultHttpGateway[source]

Default Http gateway implementation used by sources

This implementation has the following properties:

Uses cloudscraper package, which detects Cloudflare’s anti-bot pages.
```
self.session = cloudscraper.create_scraper(ssl_context=ctx)
```

Disables SSL protection, as this seems to break most sites.

ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

self.session = ... # initialize scraper session

self.session.verify = False

As such also disables InsecureRequestWarning in the request context.

with warnings.catch_warnings():
    warnings.simplefilter("ignore", InsecureRequestWarning)
    # logic