Local development, IFS file CCSID and bash startup files

[richardm90]

I'm in the process of trying out local development and working out how this can work for me when I stumbled across something that took me by surprise and is still causing me a lot of head scratching and shouts of "really!".

A bit of background:

I'm using an IBM i 7.5 server.

I'm looking at some of the open source RPG projects that have been made public to see how I can make use of them and also how others have approached the challenge of building their projects on IBM i. The one I'm currently looking at is llist project, which includes a Makefile and is well structured and thought out.

I use the Bash shell exclusively on IBM i. I can't think of a reason not to so I think it's reasonable, for any projects that I develop, that it's safe for me to assume that Bash will be available. My .profile file adds the /QOpenSys/pkgs/bin to my path and I've also installed the make-gnu and the tar-gnu packages.

What took me by surprise was that I was getting different results when I attempt to "make" the project from a Bash terminal and when I "make" the project from within VS Code.

The action I have defined in VS Code to "make" the project is as follows.

  {
    "extensions": [
      "GLOBAL"
    ],
    "name": "Build all with Makefile",
    "command": "/QOpenSys/usr/bin/make all",
    "environment": "pase",
    "deployFirst": true
  }

When I run the /QOpenSys/usr/bin/make all command from a Bash shell terminal session it fails. I understand why it fails, I need to update the Makefile compile command to include the TGTCCSID(*JOB) parameter. The files were created in the IFS using git clone and when I look at the CCSID value of the files they have a value of 1208 (UTF-8). This seemed OK to me.

The thing that took me by surprise was that when I "make" the project in VS Code it worked! I was expecting the same failure.

This is when the head scratching started.

When I looked at the CCSID value on one of the files, deployed to my VS Code build directory, I realised the IFS files had a CCSID value of 819 (ISO 8859-1 ASCII), not 1208 (UTF-8) as I'd expected.

I looked in the VS Code output tabs and found the following command in the Code for IBM i output.

/QOpenSys/pkgs/bin/tar -xof /tmp/deploy_O_kjwXDf45.tar && rm -rf PaxHeader

The deployment stage of local development creates a local tarball that is then transferred to my IBM i server and extracted to the remote deployment directory and the command did the business of extracting the files. This is using the version of tar installed using the tar-gnu package. Good, this is what I expected.

What I eventually worked out, with the help of Google and a lot more head scratching, was that the commands executed by VS Code are processed via ssh, that wasn't a surprise. The bit that did surprise me to learn was that the when commands are executed via ssh they are executed as a non-interactive non-login shell, yeah right, which basically means that the user specific startup files (except .bashrc) are not executed. This means that none of the stuff in my .profile file was being executed.

One of the environment variables I set in my .profile file is LC_ALL.

LC_ALL=EN_GB.UTF-8
export LC_ALL

The result of this is that files created in the IFS by me are created with a CCSID of 1208. I can't remember exactly when I added this or the specific reasons but it does seem to make sense especially as the open source projects are created in different locales.

If I remove the LC_ALL environment variable from my .profile and clone the repo again the files are created with a CCSID of 819 and the make command works as it did via VS Code.

I'm not sure why CCSID 819 is used. I couldn't find any definitive reference that stated this was a default value but if I change the LC_ALL it does impact on the the CCSID of files I create.

Some useful reference material:

• PASE for i Locales
• Default PASE CCSID and Locale Changed to UTF-8

Anyway, the reason I created this discussion was to get some opinions on the following.

1. Have I got this right or is there an easy way of dealing with this?

2. Should open source project files be created in the IFS with a CCSID of 1208?

• To me this seems like the right thing to do.
• The open source project compile instructions would then need to deal with this fact.

3. If that is the case then how do we best handle this when dealing with local development?

• The Code for IBM i extension uses ssh heavily so this could be an issue as it's not at all clear (at least it wasn't to me) that the environment that the commands are processed in is not the same as the users interactive Bash session.
• Add the locale into the users .bashrc file? Document this in the open source project compile instructions, probably only relevant for developers contributing to the project. The .bashrc file could be source'd in the .profile file.
• Something else?

[ThePrez]

Welcome to CCSID hell!

Here are my thoughts and opinions:

1. Have I got this right or is there an easy way of dealing with this?

The easiest way is putting your LC_ALL stuff in a .bashrc also.
One could lobby for VSCode-IBMi to use a login shell to do its work, but there are reasons for the distinction (and the separate config files) that you'd be up against.
One could also lobby for VSCode-IBMi to explicitly set the locale when it runs (problem is, we don't know what locales you have installed)

The good news is that in IBM i 7.4, 1208 would be the default PASE CCSID and some of these woes get mitigated.

2. Should open source project files be created in the IFS with a CCSID of 1208?

Yes. I have a strong opinion that everything should be UTF-8 unless there is a hard requirement for something else.

3. If that is the case then how do we best handle this when dealing with local development?

.bashrc is the correct approach, until you get to IBM i 7.4.
If this discussion resolves without a different proposal, we should open an issue to document this accordingly.

[richardm90]

Ok, it's good to know it's not me!

It's good hear your thoughts on UTF-8 for open source projects, it makes a lot of sense to me.

I'm on IBM i 7.5 but when I deploy from VS Code my IFS files are created with CCSID 819.

Is there any documentation that describes how 1208 as the default works. I did find this page which might be the documentation for this. This pages talks about the PASE locale being determined from the LANGID/CNTRYID pair. Perhaps that's what's going on for me? On my server I have QLANGID set to ENG and QCNTRYID set to GB so maybe this is causing the CCSID to be set to 819. I presume the LANGID/CNTRYID pair refers to system values as opposed to environment variables.

I'm going to press ahead and set the locale in .bashrc and see how I get on.