Recently, I was looking for Python libraries capable of executing asynchronous commands in WebDrivers, such as ChromeDriver or GeckoDriver. Unfortunately, none of them worked, so I decided to create my own library. Hence, Caqui was born.
I’ve used it in another personal project to search for over 2,000 job openings on over 200 recruitment websites. I reduced the time by at least 30% from the synchronous version to the asynchronous one, just by using three concurrent instances of WebDrivers. I still managed to create Caqui as an agnostic library that, in addition to executing commands in WebDrivers, runs commands in Winium, and tests Windows applications without installing anything additional. I’ll tell you a little more about it.
I experimented with Jmeter to test a Desktop application a few years ago. I talk more about the experiment in this article, so I won’t go on too long here. But basically, the experiment was to run Winium commands remotely using REST API to test the Windows Calculator. I used this knowledge to make Caqui run commands on Winium and WebDrivers, as described above. For this to work, it was necessary that the WebDriver be raised as a server, and its URL be used in your product, for example, a crawler.
Let’s take ChromeDriver as an example. Did you know you can run it as an ordinary shell script? By doing this, a server is raised, and it listens for requests on port 9999.