Efim.dev/log
653 m
Auto-translated from Russian
Bug #3: When Windows line endings caused chaos in the Unix world

Hello everyone! Today, I'm sharing the third story from my developer's notebook. And again, we're mentally transported back to the not-so-distant year of 2020. The project, like the last time, was built on a familiar stack: the trusty .NET, the powerful Entity Framework Core, and the reliable Npgsql for connecting to PostgreSQL.

I was working on a new feature that required executing a parameterized SQL query against the database. EF Core has a handy tool for this: FromSql. You write your SQL, and the framework automatically converts the results into neat objects. And everything was going great: it worked perfectly locally, and on the test environment in Kubernetes, where Docker images were deployed from the build server, everything seemed fine.

The Mystery from Out of Nowhere 🕵️‍♂️

Then, out of the blue, the QA team reported: "Hey, the feature isn't working in our environment!" Their environment was a local Kubernetes cluster and Docker images built directly on their machines. That was strange, because the build scripts were identical on both the build server and locally. I was quite puzzled. 🤔

There was nothing I could do but roll up my sleeves and dive into the fascinating world of debugging. And not just any debugging, but debugging directly inside a running Docker container. It's like opening up a working mechanism to figure out which gear is stuck. And after some trial and error, the picture started to become clear, like a good detective series:

1. The QA team members were, of course, using Windows.
2. Git was configured in a tricky way: when you checkout, it would automatically convert line endings to Windows style (CRLF).
3. The Docker images for their local environments were built from these local source files with CRLF line endings.
4. The SQL query in the code was written as a multi-line verbatim string literal (where you use the @ symbol before the string). So, it was compiled with Windows line endings.
5. But their local environment was running in containers on Unix, which uses LF.
6. The parser in EF Core/Npgsql, which validates the SQL query, expected either arguments immediately after the SELECT keyword, or a "correct" platform-specific line ending (Environment.NewLine), if the arguments were on separate lines.
7. Our query looked like this:
SELECT
ARG1,
ARG2,
...
8. And that's where the problem lay. 🐕 Because the code was compiled with CRLF (Windows-style), but was running in a Unix environment, the parser didn't see the "correct" LF line ending after the SELECT. It saw CRLF and thought: "What is this? An error! The query is invalid!" 💥
9. The (not so) brilliant solution: remove the line break immediately after SELECT:
SELECT ARG1,
ARG2,
...

And everything started working!

The Moral of the Story:

1. Source files with Windows line endings, compiled and run on Unix, can cause unexpected surprises. Keep an eye on core.autocrlf in Git!
2. External dependencies (hello, SQL parser!) sometimes behave unexpectedly, especially at the boundaries between systems.
3. Don't be afraid to dive into a Docker container with a debugger. It's powerful! 💪
4. Sometimes, a seemingly harmless line break for code readability can become the cause of headaches and bugs. Formatting is important, but context is everything!

Have you ever had similar adventures with line endings or tricky dependencies?

#ru #en #bugs