Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to handling lots of columns of input data for som training #21

Open
JimmyGao0204 opened this issue Mar 20, 2017 · 1 comment
Open

Comments

@JimmyGao0204
Copy link

JimmyGao0204 commented Mar 20, 2017

Recently, I want to construct a som model using somtoolbox in matlab. I have 350 SST (sea surface temperature) images. Each image is 400 pixels by 480 pixels. And each SST image was transformed into a single row vector by concatenating image rows. So that the input matrix consisted of 350 rows (image) and 192000 columns (pixels). But when I run the following code,some errors occured. Could you please tell me how to solve this problem? Thanks for your consideration.

`clc;clear;
%prepare your data (satellite remote sensing data)
myFiles=dir('D:/MatlabTemp/SSC/.tif');
fileLength=length(myFiles);
myMatrix=[];
firstRaster=imread('D:/MatlabTemp/SSC/m20030530.tif');
[m,n]=size(firstRaster);
for i=1:(fileLength)
filename=myFiles(i).name;
path='D:\MatlabTemp\SSC';
imgPath=strcat(path,'',filename);
raster=imread(imgPath);
vector=reshape(raster,1,m
n);
myMatrix=[myMatrix;vector];
end

%TSM data n * m (n samples(image)) m (pixels)
myMatrix(isnan(myMatrix)) = -1;

%inialize parameters of som algorithm
msize = [3 4];
init = 'lin';
train = 'batch';
neigh = 'ep';
rad = [5 1];

y = myMatrix;

%construct the input data used for som algorithm
sD = som_data_struct(y);

if strcmp(init,'rand')
sM = som_randinit(sD,'msize',msize);
elseif strcmp(init,'lin')
sM = som_lininit(sD,'msize',msize,'lattice','rect','shape','sheet');
end
%train som algorithm
if strcmp(train,'batch')
sM = som_batchtrain(sM, sD,'radius',rad,'trainlen',1000,'alpha_type','inv','neigh',neigh);
elseif strcmp(train,'seq')
sM = som_seqtrain(sM, sD,'radius',rad,'trainlen',1000,'neigh',neigh);
elseif strcmp(train,'sompak')
sM = som_sompaktrain(sM,sD,'radius',rad,'trainlen',1000,'neigh',neigh,'alpha_type','inv');
end`

Error using zeros

Requested 192000×192000 (274.7GB)array exceeds maximum array preference.Creation of arrays greater than this limit may take a long time and cause MATLAB to become unresponsive. See array size limit or preference panel for more information.

Error in som_liminit(line 222)
A = zeros(dim);

@lperkk
Copy link

lperkk commented Oct 15, 2024

Hello! Now, I am also experiencing this issue: Requested array exceeds maximum array preference.
Have you solved this issue please? If solved can you give me some advice or guidance?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants