vseg probabilities for hstrip

[oyahyaoui]

Dear
I am looking to understand
vseg_probabilities_for_hstrip(IplImage *y, IplImage *cropped_gradient, IplImage *downsampled_normed, IplImage *as_float)
specially applym_befe75da(vseg_model_input)
Would you mind give me a few words to start with? I am totally lost in model_befe75da.cpp, I didn't understand how did you generate this cpp file.
Many thanks in advance
Best regards.

[dgoldman-pdx]

@oyahyaoui thanks for your interest in card.io. Your questions suggest that you've already figured out a great deal of what's going on!

vseg stands for vertical segmentation. This is the process of determining the vertical position of the card number on the image of the credit card.

The vseg_probabilities_for_hstrip() function is called repeatedly, for a single scan line at a time, to determine the most likely horizontal band in which the card number resides. (hstrip == horizontal strip)

The file model_befe75da.cpp represents a deep-learning neural net model. Its function applym_befe75da() takes as its input a single scan line from the credit card image (i.e., a rectangle 1 pixel tall); its output is a vector of 3 probabilities. These 3 probabilities (which sum to 1.0) represent the likelihood that the scan line is part of a VISA-like card number (16 digits in 4 groups), part of an American Express-like card number (15 digits in 3 groups), or neither.

I would strongly suggest viewing model_befe75da.cpp (and each of the other models_*.cpp files in card.io) as a black box. Each model_*.cpp file was auto-generated from a Python object created via a collection of tools we have developed. These tools make use of Python, Cython, and Theano. Each model was trained on a large collection of sample credit card images that we have collected; the models differ from each other in their architecture and various parameters, as well as in their purpose (e.g., the above model for vertical segmentation, other models for "horizontal segmentation" of individual card number digits, and still other models for character recognition).

[oyahyaoui]

Thanks for your answer.
I am sure that the great work you put out here can be extended to all credit-card-shaped cards. I am trying to export it to build "ID cards" scanner. That's why I am trying to understand how did you auto-generated these models.
Would you mind sharing strips_sharded_crop_grad_d2_norm_n50_b150_mlp.model.pkl for example?

[dgoldman-pdx]

@oyahyaoui I could send you strips_sharded_crop_grad_d2_norm_n50_b150_mlp.model.pkl, but it wouldn't do you any good. It consists solely of a large collection of numbers. We have a Python script that reads those numbers and then exports modelm_befe75da.hpp and modelm_befe75da.cpp.

The part you really need is the entire toolchain used to create strips_sharded_crop_grad_d2_norm_n50_b150_mlp.model.pkl in the first place.

We'd love to make that toolchain public, but right now it's a horrible mishmash of complex and undocumented code (even more complex and undocumented than card.io-dmz!), and we don't currently have the time or personnel to help others make sense of everything in there.

More critically, the toolchain is designed specifically to work with our collection of credit card images. This collection is the key piece for training any new machine learning models. But we can't make those images public, because they represent actual cards from actual people.

My dream (well, one of my more prosaic dreams) is to create an app that interested developers could use to crowdsource the creation of a new collection of anonymized images from credit-card-shaped cards. At that point it would make sense for us to clean up our toolchain and share it publicly.

(In the interim, when I have spare moments, I'm currently exploring a different approach to locating the numbers on a credit card. This approach is based purely on computer vision, without any machine learning involved. If it eventually works out, then it might replace our current vertical and horizontal segmentation code in a more generally applicable manner.)

[oyahyaoui]

@dgoldman-ebay The idea of this open crowdsourcing image collection app is really a good lead to have a worldwide collection of VISA cards.
Can I help with brainstorming and trying out your new approach of finding number in a credit card?
Meanwhile I am working on my dream to find a way to have a reliable ID scanner.
Thanks

[dgoldman-pdx]

Thanks for your interest, @oyahyaoui!

Unfortunately, for at least the next few weeks, it looks like most of my time will be devoted to a project other than card.io. I hope to return fairly soon to working on the above ideas.

[WeiGoGo]

Dear author dgoldman-ebay,
I'm interested Card.iO SDK and i'm trying to understand the training data until i found this discussion.
as you said the training data is generated by collection tools and i have tried several types of the credit card with stereo credit card number for card.io sdk Demo project (looks like this: http://creditcard.calculatorsaustralia.com.au/sites/default/files/credit-card-calculator-image.png)
it's works.

Unfortunately, there is a type of credit card with "plain credit card number" (looks like this: http://pic.pimg.tw/cash9999/4a6587085d83b.jpg)
I am doubt this kinds of card which was not been trained into the training data. so that it doesn't work for that...
Would you mind to add this type of credit card into the database and generate a new model_*.cpp for me?
Because i have no idea to enhance the database...
I'm looking forward card.io and thanks for your kindness reply in advance.

Many Thanks.

Best Regards

[dgoldman-pdx]

@WeiGoGo thanks for your interest in enhancing card.io.

Adding non-embossed credit cards is indeed something that we would like to do, and we have begun working toward doing so.

However, this is a very large task, which will involve collecting appropriate videos of many such sample cards, considerable experimentation with the algorithms for detecting the card numbers from these samples, and, finally, working on the machine-learning aspect of identifying the digits. We do not expect to accomplish all this in the near future.