Timeout-based checks in VS Code extension tests
Check out all the posts in our VS Code extension development series:
When testing VS Code extensions, you might run into scenarios where you want to test a condition which is not expected to be true immediately and you cannot know when that condition will be met. For instance: an extension creates a file on disk after a certain action has been executed, and there is no way for a test to know when the extension is done creating the file.
We have faced the same problem in our own extension for Symflower and we have developed a helper function for it:
/**
* checkConditionOverTimeWithValue checks if the given condition becomes true within a certain amount of time.
* @param condition returns if the condition to check has been fulfilled and a value to return from the call.
* @returns whether a check has succeeded within time and the last return value from the condition function.
*/
export async function checkConditionOverTimeWithValue<T>(condition: () => Promise<[boolean, T]>): Promise<[boolean, T]> {
const retries = 10;
const retryTimeout = 1000;
for (let i = 0; ; i++) {
const [ok, r] = await condition();
if (ok) {
return [true, r];
} else if (i >= retries) {
return [false, r];
}
await timeout(retryTimeout);
}
}
As mentioned in the API documentation, it checks a given condition multiple times, waiting for one second between each check. We have found that value to be reasonably fast and not stress our CI runners too much. The implementation uses another small helper function called timeout which is just a promisified version of the usual setTimeout function.
That’s great news: we’re hiring!
/** timeout returns a promise that resolves after the given amount of time and optionally returns a value when passed as a second argument. */
export const timeout = promisify(setTimeout);
The second, generic return value (type T) of checkConditionOverTimeWithValue is particularly useful for building assertion failure messages without having to retrieve a value again that was checked in the condition function. For example:
export async function assertEqualFileContent(filePathA: string, filePathB: string) {
const [ok, lastDiffResult] = await checkConditionOverTimeWithValue(async () => {
const result = await diffOfFiles(filePathA, filePathB);
return [result.equal, result];
});
if (!ok) {
assert.fail(`Expected no diff between ${filePathA} and ${filePathB} but got:\n\n${lastDiffResult.diffOutput}\n`);
}
}
However, not all applications of the helper function have a return value. Some use cases only check a boolean condition instead of performing some kind of comparison of a value. For those cases, we have another helper called checkConditionOverTime:
export async function assertFileExists(path: string) {
assert.isTrue(await checkConditionOverTime(async () => existsSync(path)), `Expected file "${path}" to exist but file did not exist`);
}
Which simply wraps checkConditionOverTimeWithValue and discards the second value:
/**
* checkConditionOverTime checks if the given condition becomes true within a certain amount of time.
* @param condition returns if the condition to check has been fulfilled.
* @returns whether a check has succeeded within time.
*/
export async function checkConditionOverTime(condition: () => Promise<boolean>): Promise<boolean> {
return (await checkConditionOverTimeWithValue(async () => [await condition(), undefined]))[0];
}
That’s great news: we’re hiring!
Found something useful in this article? Great! Perhaps you would also enjoy Symflower for IntelliJ IDEA, GoLand, Android Studio, and of course VS Code to help with writing and maintaining software tests. You’re also invited to join our newsletter where we post insights into software development and testing.
If you have any questions or feedback for our articles or our extension, we’d love to hear from you! You can send us an hello@symflower.com or find us on social media.