技巧
超时
每个请求可以有一个最大允许运行时间。
为了使用此功能,请指定request
超时选项。
| import got from "got";
const body = await got("https://httpbin.org/anything", {
timeout: {
request: 30000,
},
});
|
有关更具体的超时,请访问超时 API.
重试
默认情况下,如果可能,Got会对失败的请求进行新的重试。
通过将允许的最大重试次数设置为0
,可以完全禁用此功能。
| import got from "got";
const noRetryGot = got.extend({
retry: {
limit: 0,
},
});
|
要指定可检索的错误,请使用重试API.
Cookies
Got 支持 cookies 开箱即用。
不需要手动解析它们。
为了使用cookie,从tough-cookie
包中传递一个CookieJar
实例。
| import got from "got";
import { CookieJar } from "tough-cookie";
const cookieJar = new CookieJar();
await cookieJar.setCookie("foo=bar", "https://httpbin.org");
await got("https://httpbin.org/anything", { cookieJar });
|
AWS
对AWS服务的请求需要对其标头进行签名。
这可以通过使用 got4aws
包来完成。
这是一个使用签名请求查询API 网关
的示例。
| import got4aws from "got4aws";
const got = got4aws();
const response = await got("https://<api-id>.execute-api.<api-region>.amazonaws.com/<stage>/endpoint/path", {
// …
});
|
分页
在处理大型数据集时,使用分页是非常有效的
默认情况下,Got使用Link
头来检索下一页。
但是,这种行为可以自定义,参见分页API。
| const countLimit = 50;
const pagination = got.paginate("https://api.github.com/repos/sindresorhus/got/commits", {
pagination: { countLimit },
});
console.log(`Printing latest ${countLimit} Got commits (newest to oldest):`);
for await (const commitData of pagination) {
console.log(commitData.commit.message);
}
|
UNIX域套接字
参考enableUnixSockets
选项.
测试
Got使用本机http
模块,该模块依赖于本机net
模块。
这意味着有两种可能的测试方法:
- 使用像[
nock
]这样的mock库(https://github.com/nock/nock),
- 创建服务器。
第一种方法应该涵盖所有常见的用例
请记住,它覆盖了本地的http
模块,因此可能会由于差异而出现bug。
最可靠的方法是创建一个服务器
在某些情况下,nock
可能不够或缺乏功能。
Nock
By default nock
mocks only one request.\
Got will retry on failed requests by default, causing a No match for request ...
error.\
The solution is to either disable retrying (set options.retry.limit
to 0
) or call .persist()
on the mocked request.
默认情况下,nock
只模拟一个请求。
默认情况下,Got将对失败的请求进行retry,导致No match for request ...
的错误。
解决方案是禁用重试(将options.retry.limit
设置为0
)或在模拟请求上调用.persist()
。
| import got from "got";
import nock from "nock";
const scope = nock("https://sindresorhus.com").get("/").reply(500, "Internal server error").persist();
try {
await got("https://sindresorhus.com");
} catch (error) {
console.log(error.response.body);
//=> 'Internal server error'
console.log(error.response.retryCount);
//=> 2
}
scope.persist(false);
|
代理
虽然没有一个完美的、没有错误的软件包,但Apify解决方案是一个现代的解决方案
查看got-scraping/src/agent/h1-proxy-agent.ts
。
它具有与hpagent
相同的API。
hpagent
is a modern package as well. In contrast to tunnel
, it allows keeping the internal sockets alive to be reused.
| import got from "got";
import { HttpsProxyAgent } from "hpagent";
await got("https://sindresorhus.com", {
agent: {
https: new HttpsProxyAgent({
keepAlive: true,
keepAliveMsecs: 1000,
maxSockets: 256,
maxFreeSockets: 256,
scheduling: "lifo",
proxy: "https://localhost:8080",
}),
},
});
|
Alternatively, use global-agent
to configure a global proxy for all HTTP/HTTPS traffic in your program.
If you're using HTTP/2, the http2-wrapper
package provides proxy support out-of-box.\
Learn more.
在没有代理的情况下重试
如果使用代理,可能会遇到连接问题。
一种解决方法是在重试时禁用代理。
流API的解决方案是这样的:
| import https from "https";
import fs from "fs";
import got from "got";
class MyAgent extends https.Agent {
createConnection(port, options, callback) {
console.log(`Connecting with MyAgent`);
return https.Agent.prototype.createConnection.call(this, port, options, callback);
}
}
const proxy = new MyAgent();
let writeStream;
const fn = (retryStream) => {
const options = {
agent: {
https: proxy,
},
};
const stream = retryStream ?? got.stream("https://example.com", options);
if (writeStream) {
writeStream.destroy();
}
writeStream = fs.createWriteStream("example-com.html");
stream.pipe(writeStream);
stream.once("retry", (retryCount, error, createRetryStream) => {
fn(
createRetryStream({
agent: {
http: undefined,
https: undefined,
http2: undefined,
},
})
);
});
};
fn();
|
h2c
没有直接的h2c
支持。
然而,你可以在beforeRequest
钩子中提供一个h2session
选项。
参见例子。
大写标头
Got总是将标头规范化,因此传递Uppercase-Header
会将其转换为uppercase-header
。
要解决这个问题,你需要传递一个包装代理:
| class WrappedAgent {
constructor(agent) {
this.agent = agent;
}
addRequest(request, options) {
return this.agent.addRequest(request, options);
}
get keepAlive() {
return this.agent.keepAlive;
}
get maxSockets() {
return this.agent.maxSockets;
}
get options() {
return this.agent.options;
}
get defaultPort() {
return this.agent.defaultPort;
}
get protocol() {
return this.agent.protocol;
}
}
class TransformHeadersAgent extends WrappedAgent {
addRequest(request, options) {
const headers = request.getHeaderNames();
for (const header of headers) {
request.setHeader(this.transformHeader(header), request.getHeader(header));
}
return super.addRequest(request, options);
}
transformHeader(header) {
return header
.split("-")
.map((part) => {
return part[0].toUpperCase() + part.slice(1);
})
.join("-");
}
}
const agent = new http.Agent({
keepAlive: true,
});
const wrappedAgent = new TransformHeadersAgent(agent);
|
参看例子.
自定义选项
当一个选项不存在时 Got v12 抛出。因此传递一个顶级选项,如:
| import got from "got";
await got("https://example.com", {
foo: "bar",
});
|
将抛出。为了防止这种情况,你需要在init
钩子中读取该选项:
| import got from "got";
const convertFoo = got.extend({
hooks: {
init: [
(rawOptions, options) => {
if ("foo" in rawOptions) {
options.context.foo = rawOptions.foo;
delete rawOptions.foo;
}
},
],
},
});
const instance = got.extend(convertFoo, {
hooks: {
beforeRequest: [
(options) => {
options.headers.foo = options.context.foo;
},
],
},
});
const { headers } = await instance("https://httpbin.org/anything", { foo: "bar" }).json();
console.log(headers.Foo); //=> 'bar'
|
最后,您可能想要创建一个捕获所有实例:
| import got from "got";
const catchAllOptions = got.extend({
hooks: {
init: [
(raw, options) => {
for (const key in raw) {
if (!(key in options)) {
options.context[key] = raw[key];
delete raw[key];
}
}
},
],
},
});
const instance = got.extend(catchAllOptions, {
hooks: {
beforeRequest: [
(options) => {
// All custom options will be visible under `options.context`
options.headers.foo = options.context.foo;
},
],
},
});
const { headers } = await instance("https://httpbin.org/anything", { foo: "bar" }).json();
console.log(headers.Foo); //=> 'bar'
|
不支持Electron net
模块
Note
得到了v12和更高版本的ESM包,但是Electron还不支持ESM。所以你需要使用Got v11。
Got doesn't support the electron.net
module. It's missing crucial APIs that are available in Node.js.\
While Got used to support electron.net
, it got very unstable and caused many errors.
However, you can use IPC communication to get the Response object:
| // Main process
const got = require("got");
const instance = got.extend({
// ...
});
ipcMain.handle("got", async (event, ...args) => {
const { statusCode, headers, body } = await instance(...args);
return { statusCode, headers, body };
});
// Renderer process
async () => {
const { statusCode, headers, body } = await ipcRenderer.invoke("got", "https://httpbin.org/anything");
// ...
};
|