Scandinavian characters, CCSID and PASE CCSID/Locale

[krka01]

Hi,

We are not able to get Code for IBM i to work with our special scandinavian charachters ÅÄÖ.

Our systems have been around for long time, since the S/36 time so unfortunately our system value QCCSID is set to 65535.
Changing the user profiles to use 1143 Swedish/Finnish with Euro does not solve the issue.

I have read the https://halcyon-tech.github.io/docs/#/pages/tips/ccsid and here it gets interesting.

When we upgraded to IBM i 7.4 and IBM made changes to default PASE CCSID and Locale to default to UTF-8 as outlined in
https://www.ibm.com/docs/en/i/7.4?topic=system-default-pase-ccsid-locale-changed-utf-8
we did run into a lot of problems as files were created with CCSID 1208 instead of 819.

To circumvent this problem we had to use
ADDENVVAR ENVVAR(PASE_DEFAULT_UTF8) VALUE(N) LEVEL(*SYS)
as outlined in
https://www.ibm.com/support/pages/after-v7r4-upgrade-encoding-scheme-not-compatible-utilizing-gifsgatecpyfrmstmf-gentran-server-iseries

The correct approach would be to find and fix everything so we can use PASE with UTF-8, but I do not see this will be done in years.

Does anybody know of a work around for this?

Thanks in advance

[pabloto]

Hi,

You have change the sys value correctly, but probably your pfsrc have the ccsid set to 65535. Is it true? Try with dspfd of your source file and look at ccsid of this.

Bye

[krka01]

Hi,

Thanks for the reply. Our source files have CCSID = 278 Swedish/Finnish without Euro sign.

The problem is that we can not have PASE to default to UTF-8 for CCSID and Locale. I suspect we can not start the SSH server with UTF-8 either as several of the files we had problems with was transfered to the IBM i using sftp.

[worksofliam]

Hi!

Most importantly you need to fix the PASE_LANG, QIBM_PASE_CCSID and either QCCSID or your user profile CCSID. I am afraid your best bet is disabling SQL and using the non-sql methods which will not face many CCSID issues.

Another suggestion is running a second SSH Daemon with those environment variables setup if possible.

[krka01]

Hi,

Thanks for the reply. I have set the CCSID on my profile and that did not help. SQL is disabled for the connection, but it still does not work.

[worksofliam]

@krka01 Feel free to share how they look in the UI. Also still share the output when you see the invalid characters.

[krka01]

Yes, I will, but need to postpone to next week as I have left the office for the weekend.

I will also check the possibility to run a second ssh server on another port. I have already created one, but need to create a small CL programs that will set the right environmental variables and have the network people open the port in the firewall.

[chrjorgensen]

@krka01 You could use Service Commander for IBM i to control this second SSH server, having the environment variables set by Service Commander for IBM i when starting the service.

Service Commander for IBM i can be installed from the RPM's using yum install service-commander or through ACS Open Source Package Management. This is a great tool for handling services and can even be integrated with the STRTCPSVR and ENDTCPSVR commands!

This way you don't need to code any CL program to control the second SSH service - SC can do that for you.

[krka01]

Hi,

I have now setup an second SSH Daemon listening on port 2222 and make sure that the PASE environment variables are correctly set before starting the second SSHD and it seems like the Scandinavian characters are now handle correctly.

For further reference, just changing the port(SSH) on the Code for IBM i Login Setting, remove all previously created filters and open the connection/sign in again was not enough, I had to delete the connection and add it from scratch to get it to work properly.

@chrjorgensen I am aware of the Service Commander and it seems like a great tool, but it seemed overkill to use it for this specific case, it just needed a small ( 4 rows) CL program to accomplish this.

Thanks for your help and keep up the good work.

[dkJohan]

Hi krka01, could you copy the cl progam line here please? Maybe we can do something in an easyer way

[krka01]

Hi,

First you need to decide if you want to start the secondary SSH daemon with the -p port option or use a dedicated config file.
I choosed to start with an dedicated config file as I think it is more flexible to configure the secondary ssh daemon to only be accessible for certain groups using the AllowGroups and so on.
Decide the port to use, I choosed port 2222

To create a copy of the original sshd_config:

CPY OBJ('/QOpenSys/QIBM/UserData/SC1/OpenSSH/etc/sshd_config')
    TOOBJ('/QOpenSys/QIBM/UserData/SC1/OpenSSH/etc/sshd_config_2222')
    OWNER(*KEEP) AUT(*OBJ) 

Edit the new config file and uncomment and change the port directive to use the choosen port:
EDTF '/QOpenSys/QIBM/UserData/SC1/OpenSSH/etc/sshd_config_2222'

Create a user profile for the secondary daemon job:
CRTUSRPRF USRPRF(SSHDUSR) PASSWORD(*NONE) INLMNU(*SIGNOFF) LMTCPB(*YES) SPCAUT(*ALLOBJ) + TEXT('SSHD Daemon user profile')

Create the CLLE program MYLIB/MYSSHD:

Pgm

/* For version 7.4 and later */
/* PASE_DEFAULT_UTF8  does not exist prior 7.4 */
AddEnvVar EnvVar(PASE_DEFAULT_UTF8) Value(Y) Level(*JOB) + 
          Replace(*YES) 

/* For version 7.3 and earlier */		  
AddEnvVar EnvVar(QIBM_PASE_CCSID) Value(1208) Level(*JOB) +        
          Replace(*YES)                                            

/* All versions, change to the correct PASE Locale */                                                                    
AddEnvVar EnvVar(PASE_LANG) Value(FI_FI) Level(*JOB) +             
          Replace(*YES)                                            

/* Start the SSH daemon with the new config file */                                                                   
Qsh Cmd('/QOpenSys/usr/sbin/sshd -f +                              
         /QOpenSys/QIBM/UserData/SC1/OpenSSH/etc/sshd_config_2222')

EndPgm

To start the secondary daemon:

SBMJOB CMD(CALL PGM(MYLIB/MYSSHD))
       JOB(SSHDUSR)
       JOBD(QGPL/QDFTSVR)	   
       JOBQ(QSYS/QUSRNOMAX)      
       OUTQ(QPRINT)                
       USER(ABALLOBJ)               
       RTGDTA(*JOBD)                
       SYSLIBL(*SYSVAL)             
       CURLIB(*USRPRF)              
       INLLIBL(*SYSVAL)             
       INLASPGRP(*JOBD)
       ALWMLTTHD(*YES)

If system value QCCSID is 65535 and want to have SQL enabled for Code for IBM i, set the correct CCSID on the user profiles
or have the user change it by them self with the CHGPRF CCSID(<correct_ccsid>) command.

The secondary ssh daemon can be configured to start as a autostart job for an existing subsystem or in an dedicated subsystem.
The advantage to use a dedicated subsystem is that you can start and stop the daemon with STRSBS and ENDSBS.

Reference:
https://www.ibm.com/support/pages/starting-ssh-daemon-dedicated-subsystem-environment

https://www.ibm.com/docs/en/i/7.4?topic=ssw_ibm_i_74/apis/pase_locales.html

https://man.openbsd.org/sshd