It’s Time to Say GoodBye to pd.read_csv() and pd.to_csv()
<p>Input-output operations with Pandas to a CSV are serialized, making them incredibly inefficient and time-consuming. It's frustrating when I see ample scope for parallelization here, but unfortunately, Pandas does not provide this functionality (yet). Although I am never in favor of creating CSVs in the first place with Pandas (read my post below to know why), I understand that there might be situations where one has no other choice but to work with CSVs.</p>
<h2><a href="https://towardsdatascience.com/why-i-stopped-dumping-dataframes-to-a-csv-and-why-you-should-too-c0954c410f8f?source=post_page-----27fbc74e84c5--------------------------------" rel="noopener follow" target="_blank">Why I Stopped Dumping DataFrames to a CSV and Why You Should Too</a></h2>
<h3><a href="https://towardsdatascience.com/why-i-stopped-dumping-dataframes-to-a-csv-and-why-you-should-too-c0954c410f8f?source=post_page-----27fbc74e84c5--------------------------------" rel="noopener follow" target="_blank">It’s time to say goodbye to pd.to_csv() and pd.read_csv()</a></h3>
<p><a href="https://towardsdatascience.com/why-i-stopped-dumping-dataframes-to-a-csv-and-why-you-should-too-c0954c410f8f?source=post_page-----27fbc74e84c5--------------------------------" rel="noopener follow" target="_blank">towardsdatascience.com</a></p>
<p>Therefore, in this post, we will explore <strong>Dask</strong> and <strong>DataTable</strong>, two of the most trending Pandas-like libraries for Data Scientists. We’ll rank Pandas, Dask and Datatable based on their performance on the following parameters:</p>
<ol>
<li><strong>Time taken to read the CSV and obtain a PANDAS DATAFRAME</strong></li>
</ol>
<p>If we read a CSV through Dask and DataTable, they will generate a Dask DataFrame and DataTable DataFrame respectively, <strong>not the Pandas DataFrame</strong>. Assuming that we want to stick to the traditional Pandas syntax and functions (due to familiarity), we would have to convert these to a Pandas DataFrame first, as shown below.</p>
<p><a href="https://towardsdatascience.com/its-time-to-say-goodbye-to-pd-read-csv-and-pd-to-csv-27fbc74e84c5"><strong>Click Here</strong></a></p>