Describe the bug
When using toSeparated (or toCSV/toTSV), nullable columns are not handled correctly. toSeparated uses showElement, which ignores the column's null mask and directly gets the value from the column vector. For unboxed columns, this returns an incorrect default value, for boxed vectors, it throws a null pointer exception.
To Reproduce
{-# LANGUAGE OverloadedStrings #-}
import DataFrame qualified as D
dfUnboxed = D.fromNamedColumns [("a", D.fromList [Nothing, Just 1 :: Maybe Int])]
dfBoxed = D.fromNamedColumns [("a", D.fromList [Nothing, Just [] :: Maybe [Int]])]
λ> D.toSeparated ',' dfUnboxed
"a\n0\n1\n"
λ> D.toSeparated ',' dfBoxed
"*** Exception: fromMaybeVec: Nothing slotExpected behavior
toSeparated should check the nullable mask and return a null token (empty string?) for null cells, similar to show.
Describe the bug
When using
toSeparated(ortoCSV/toTSV), nullable columns are not handled correctly.toSeparatedusesshowElement, which ignores the column's null mask and directly gets the value from the column vector. For unboxed columns, this returns an incorrect default value, for boxed vectors, it throws a null pointer exception.To Reproduce
{-# LANGUAGE OverloadedStrings #-} import DataFrame qualified as D dfUnboxed = D.fromNamedColumns [("a", D.fromList [Nothing, Just 1 :: Maybe Int])] dfBoxed = D.fromNamedColumns [("a", D.fromList [Nothing, Just [] :: Maybe [Int]])] λ> D.toSeparated ',' dfUnboxed "a\n0\n1\n" λ> D.toSeparated ',' dfBoxed "*** Exception: fromMaybeVec: Nothing slotExpected behavior
toSeparatedshould check the nullable mask and return a null token (empty string?) for null cells, similar toshow.