Import a CSV file

A CSV import reads a file you upload and writes its rows into a data table. It runs in the background in small batches, so even a very large file never times out or blocks your site.

Open the table, go to the Import & feed tab, and use the CSV import card: "Upload a CSV file whose first row holds the column names. Large files are imported in the background."

Before you start

• The table needs columns. Without columns the tab says Add columns first.
• The first row of the file must hold column names. Names are only used to suggest a match; they do not have to equal the table's keys.
• The file must be a .csv or .txt file, at most 50 MB by default (wpmcf/data_tables/import_max_bytes). Your server's own upload limit (upload_max_filesize and post_max_size in PHP) also applies. For bigger files raise both, or split the file.
• To update existing rows you need a unique column. See Create a data table.
• Only one import into a table can run at a time.

Step by step

1. Click Import CSV. The dialog Import into "Table name" opens.
2. Click Choose CSV file and pick the file. The text above the button says: "Choose a CSV file. The first row must contain column names; comma, semicolon and tab separators are detected automatically." A vertical bar | is detected too.
3. The dialog now shows three blocks. Nothing has been written yet.
   
   Match columns. One line per table column, with a picker CSV column for {label}. The file's header names fill the pickers: a column is matched when its key or label equals a header, ignoring capitals, spaces, dashes and other punctuation (Unit Price matches unit_price). Choose - Skip - for a column the file does not fill. A header that is empty is shown as Column 3 (its position). With more than 12 CSV columns the picker gets a Filter columns... box.
   
   First rows of the file. The first five rows, so you can check that the columns and the text (accents, quotes) were read correctly.
   
   Existing rows. What to do with rows already in the table (see below).
4. Click Import ~N rows. The number is an estimate from the file size. The button stays disabled until at least one column is matched (and, for update, a match column is chosen).
5. A progress bar shows Import progress and Importing... N rows processed. You can close this window; the import continues in the background. Click Close whenever you like, or watch until the message ends with Done.

When it finishes you see N rows imported, M failed. If some rows failed, the box Some rows could not be imported lists the first five database messages, for example Duplicate entry 'I1' for key '...uq_sku'.

Existing rows

The Existing rows options decide what happens to rows already in the table:

With Update and Replace the Match rows by column must be one of the columns you matched. In Update mode, on the columns you matched, an empty cell empties the stored value. Columns you did not match (skipped) are left alone.

Replace: two ways

• Replace with a match field. Choose a unique column in Match rows by (recommended).... Matching rows are updated, new rows added, and rows missing from the file deleted at the end. Row IDs are kept, so anything that remembers a row by its ID (generated pages with their URLs, AI content and hand edits) is kept as well. The dialog says "Rows are matched by this field: matching rows are updated, new rows added and rows missing from the new data deleted. Generated pages keep their URLs, AI content and hand edits."
• Replace without a match field (Do not match: replace all rows). The new rows are first loaded into a working copy of the table. Only after every row was read without errors is the working copy swapped in, in one step; until then the live table is untouched. If the file has no rows, or any row fails, nothing changes and the message says The new data has no rows, so the data table was left unchanged. or N rows could not be written, so the data table was left unchanged. Fix them and try again. Rows that did not change keep their ID; a changed row keeps it only when at least half of its columns are unchanged, otherwise it gets a new ID. The dialog warns: "Unchanged rows keep their identity, but a changed row may get a new one, so its generated page can lose its AI content and hand edits. Match rows by a unique field to avoid this."

When rows fail during a replace with a match field, rows missing from the file are not deleted (they cannot be told apart from rows that failed): Rows missing from the file were kept because some rows failed; fix them and import again to remove the rest.

How values are read

An import is more forgiving than typing a row, because files are often messy. Each cell goes through the column's type:

Quoted cells with commas, quotes ("") and line breaks inside are read correctly.

Separators and encodings

• Separator: the first line decides: comma, semicolon, tab or vertical bar. You cannot choose it by hand; a file that mixes them is best saved again from your spreadsheet as a standard CSV.
• Encoding is detected from the file:
• • UTF-8, with or without a byte-order mark (the usual "CSV UTF-8" of Excel);
  • UTF-16 little or big endian with a byte-order mark (Excel's "Unicode Text" export);
  • anything else that is not valid UTF-8 is read as a Windows code page chosen from the site language: Windows-1254 (Turkish), 1250 (Polish, Czech, Slovak, Hungarian, Romanian, Croatian, Slovenian ...), 1251 (Russian, Ukrainian, Bulgarian, Serbian ...), 1253 (Greek), 1255 (Hebrew), 1256 (Arabic, Persian, Urdu), 1257 (Baltic), 1258 (Vietnamese), otherwise 1252 (Western European).

If accents look wrong in First rows of the file (for example Ý instead of İ), the file is in a code page that does not match the site language. Save it again as CSV UTF-8 from your spreadsheet, or change the assumed code page with the wpmcf/import/fallback_encoding filter (it receives the default and the locale and returns a code page name such as Windows-1254).

Large files and the background

• The uploaded file is stored under wp-content/uploads/wpmcf-imports/ with a random name, protected from direct web access, and is deleted as soon as the import is over, whatever the outcome. A file you uploaded but never imported is removed when you close the dialog, and in any case after a day.
• The import is a task. Rows are written in chunks of 500; one batch runs for about 20 seconds (or until 80 % of PHP's memory limit) and the next batch continues exactly where the last stopped, so a restart never repeats finished rows. Batches are run by Action Scheduler. See Background tasks for what starts the queue and what to do when it seems stuck.
• You can close the dialog and leave the page. When you come back the table has the new rows. (The progress of an import you left cannot be reopened; the table is the result.)
• Starting a second import into the same table while one runs is refused: An import into this data table is still running. Wait until it is done, then start the next one.

Messages

For developers

• wpmcf/data_tables/import_max_bytes changes the size limit (default 50 MB).
• wpmcf/import/fallback_encoding changes the assumed code page.
• When an import finishes wpmcf/data_tables/synced fires with the table key and a summary (['type' => 'import', 'mode' => ..., 'rows' => ..., 'failed' => ...]). See the actions reference.