Limited to No Encoding Options

[Yin117]

I'm trying to read a file CSV with £1,000,000 in the content, however it is parsing out as �1,000,000 I believe this is due to a default behaviour and the lack of option to read as e.g. latin1 which would resolve the issue.

If there is a recommend approach for this such as reading as a Buffer then parsing, please recommend it here as a good reference for others.

[MrKampla]

I can't replicate this, I've created a csv file with '£1,000,000' as content, I've passed readAs: "Text" prop to useFilePicker and it parses the content and displays it correctly. useFilePicker is using the FileReader browser primitive under the hood which uses UTF-8 for encoding by default, which should be able to decode latin1 encoded files.

Could you provide a code example of useFilePicker usage with a csv file that produces this issue?

[Yin117]

of course, here's some resources:

File: useFilePicker_issue_102_exampleFile_01.csv

Code:

  const {
    openFilePicker,
    filesContent,
    plainFiles,
  } = useFilePicker({
    accept: '.csv',
    multiple: true,
    onFilesSelected: ({ filesContent, plainFiles }) => {
      const {
        jsonResults,
        errors,
      } = processCSVToJSON(filesContent, plainFiles, { delimiter: ',', supportQuotedFields: true });
    }
  });

Function processVSVToJSON:

export function processCSVToJSON<T>(fileContents: FileContent<T>[], plainFiles: PlainFile[], options: OptionsProcessCSVToJSON) {

  const {
    delimiter,
    supportQuotedFields,
  } = options;

  const jsonResults: JsonObj[][] = [];
  const errors: Error[] = [];

  for (let i = 0; i < fileContents.length; i++) {
    const content = fileContents[i];
    const file = plainFiles[i];

    console.log('fileContents', content);

Result of console.log

[MrKampla]

@Yin117 Thank you for providing an example, I was able to replicate this issue and I've already created a PR with a fix #103
In the next version of useFilePicker, you'll be able to specify the encoding as a prop in the main hook.

Nice catch @Yin117! Watch this thread to get notified when we ship the fix for your issue.

[Yin117]

Awesome, thanks @MrKampla

I absolutely love use file picker and I always recommend it to anyone wanting to do File Picking in React.

[MrKampla]

I'm closing this issue as the encoding selection feature just arrived in version 2.1.4