csvDelimiter and project CSV delimiter not consistently respected in exports and imports

[astruebi]

Title

csvDelimiter / project CSV delimiter not consistently respected in exports and imports (exportRecordsTyped, exportProjectInformation, exportDags, importDags, writeDataForImport)

---

Summary

When the REDCap project/user setting "Delimiter for CSV file downloads" is changed from the default comma (,) to semicolon (;) (or potentially other supported delimiters), several redcapAPI functions start to fail or misbehave:

• exportRecordsTyped() (and related export helpers)
• exportProjectInformation()
• exportDags()
• importDags()
• writeDataForImport() (used by multiple import functions)

The core problem appears to be that the package:

1. Does not consistently pass the desired csvDelimiter to the API in body, and
2. Always parses/creates CSV with comma, regardless of the project/user delimiter.

This leads to parsing errors on exports and HTTP 400 responses on imports when the project delimiter is set to ;.

---

Environment

• Package: redcapAPI
• REDCap: 14.x (exact version probably not critical, behaviour is tied to the CSV delimiter setting)
• R: recent 4.x
• REDCap project setting:
  Control Center → User Settings → "Delimiter for CSV file downloads" = ;

---

Minimal example (exports)

1. In REDCap, set:

   Control Center → User Settings → "Delimiter for CSV file downloads" = semicolon ( ; )

2. In R:

library(redcapAPI)

rcon <- redcapConnection(
  url   = "https://your.redcap.server/api/",
  token = "…"
)

# This may error or produce malformed data when delimiter is ';'
dat <- exportRecordsTyped(rcon = rcon, forms = "some_form")

Observed issues:

• In some cases:
  Error in rbind(deparse.level, ...) : numbers of columns of arguments do not match
• In other cases, fields / columns are misaligned or missing.

Root cause (from reading the code):

• exportRecords* and related helpers rely on makeApiCall() returning a CSV string, then use read.csv / as.data.frame.response without specifying sep.
• When the REDCap project is configured to use ; as delimiter, the API returns semicolon-separated CSV, but read.csv still assumes sep = ",", causing incorrect parsing.

---

Minimal example (projectInformation)

With the same project setting (; as delimiter):

rcon <- redcapConnection(url = "…", token = "…")

# Triggers exportProjectInformation.redcapApiConnection()
pi <- rcon$projectInformation()

# Somewhere later in the code:
if (rcon$projectInformation()$is_longitudinal == 0) {
  ...
}

Observed error:

Error in if (rcon$projectInformation()$is_longitudinal == 0) { : 
  argument is of length zero

Root cause:

• exportProjectInformation.redcapApiConnection() currently does:

  body <- list(content = "project",
               format = "csv",
               returnFormat = "csv")
  
  as.data.frame(makeApiCall(rcon, body, ...))

• as.data.frame.response() uses read.csv(text = mapped, ...) with the default comma separator.

• When the API returns semicolon-separated CSV, the entire header line is treated as one column; there is no is_longitudinal column, hence the argument is of length zero error.

---

Minimal example (DAG import)

Again with project delimiter set to ;:

rcon <- redcapConnection(url = "…", token = "…")

new_data_access_groups <- data.frame(
  data_access_group_name = "AXS_2025-12-17 11:09:17.164299",
  unique_group_name      = NA_character_,
  stringsAsFactors       = FALSE
)

redcapAPI::importDags(rcon = rcon, data = new_data_access_groups)

Observed behaviour:

• With comma delimiter, the call succeeds.
• With semicolon delimiter, the underlying API call returns HTTP 400.

The data frame is fine (checked via str() / inspection), so the issue is in how the CSV payload is constructed.

Root cause:

• importDags.redcapApiConnection() does:

  body <- list(content = "dag",
               action = "import", 
               format = "csv", 
               returnFormat = "csv", 
               data = writeDataForImport(data))

• writeDataForImport() currently does:

  writeDataForImport <- function(data){
    coll <- checkmate::makeAssertCollection()
    checkmate::assert_data_frame(x = data, add = coll)
    checkmate::reportAssertions(coll)
  
    output <-
      utils::capture.output(
        utils::write.csv(data,
                         file = "",
                         na = "",
                         row.names = FALSE)
      )
  
    paste0(output, collapse = " ")
  }

• write.csv() is hard-coded to use , as separator (it ignores any sep argument).

• When the REDCap project is configured to use ; as CSV delimiter, it appears REDCap applies this also to DAG imports and expects semicolon-separated input, but gets comma-separated CSV → HTTP 400.

---

Proposed changes (backwards compatible)

1. Connection-level CSV delimiter

Introduce a delimiter state on the redcapApiConnection object with a small set of helper methods:

• rcon$csv_delimiter() → returns one of c(",", " ", ";", "|", "^"), default ",".
• rcon$set_csv_delimiter(d) → validates and sets the delimiter.
• rcon$csv_delimiter_api() → maps the delimiter to the API value ("comma", "semicolon", "tab", "pipe", "caret").

Example helper:

getCsvDelimiterAPI <- function(csv_delimiter) {
  switch(
    csv_delimiter,
    ","  = "comma",
    "	" = "tab",
    ";"  = "semicolon",
    "|"  = "pipe",
    "^"  = "caret",
    "comma"
  )
}

Then in redcapConnection():

• add private state variables csv_delim and csv_delim_api,
• expose them via csv_delimiter(), set_csv_delimiter(), csv_delimiter_api().

That allows user code to do:

rcon <- redcapConnection(url = "…", token = "…")
rcon$set_csv_delimiter(";")  # or keep default ","

independent of REDCap’s UI/user setting.

---

2. Use the connection’s delimiter in CSV-based exports

For all exported CSV that the package parses back (e.g. content = "project", "dag", "record", "exportFieldNames"), pass the API delimiter and parse with the same delimiter, e.g.:

body <- list(content      = "project",
             format       = "csv",
             returnFormat = "csv",
             csvDelimiter = rcon$csv_delimiter_api())

project_response <- makeApiCall(rcon, body, ...)

as.data.frame(project_response, sep = rcon$csv_delimiter())

Similarly for:

• exportDags.redcapApiConnection()
• exportProjectInformation.redcapApiConnection()
• exportRecords.redcapApiConnection()
• .exportFieldNamesApiCall() (internal helper used e.g. for imports)

This makes request and parsing consistent for any supported delimiter.

---

3. Make writeDataForImport() delimiter-aware

Change writeDataForImport() to accept a csv_delimiter argument (default ",") and use write.table() instead of write.csv():

writeDataForImport <- function(data, csv_delimiter = ","){
  coll <- checkmate::makeAssertCollection()
  checkmate::assert_data_frame(x = data, add = coll)
  checkmate::reportAssertions(coll)

  output <-
    utils::capture.output(
      utils::write.table(data,
                         file      = "",
                         sep       = csv_delimiter,
                         na        = "",
                         row.names = FALSE,
                         col.names = TRUE,
                         qmethod   = "double")
    )

  paste0(output, collapse = " ")
}

Then, in importDags.redcapApiConnection() (and any other import* that relies on writeDataForImport()), pass the connection delimiter:

body <- list(content      = "dag",
             action       = "import", 
             format       = "csv", 
             returnFormat = "csv", 
             data         = writeDataForImport(data, csv_delimiter = rcon$csv_delimiter()))

This keeps the default behaviour (comma) for all existing code and makes it possible to use semicolon (or other) consistently when desired.

---

Why this matters

• Right now, changing the project/user CSV delimiter setting in REDCap can silently break redcapAPI for some functions, particularly:

  • exportRecordsTyped() and related helpers,
  • projectInformation() / exportProjectInformation(),
  • exportDags() / importDags().

• The package already mentions csvDelimiter usage in the documentation and examples (e.g. exporting with pipe "|"), but the handling is not consistently wired through all helpers and parsing logic.

• A connection-level CSV delimiter plus consistent usage in exports and imports would:

  • be fully backwards compatible (default comma),
  • decouple redcapAPI behaviour from the end-user GUI setting in REDCap,
  • and align the package with REDCap’s own csvDelimiter API parameter.