Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[*] improve log message for storage view content #1718

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tirivo
Copy link

@tirivo tirivo commented Jun 4, 2024

Improve print content of StorageView, limited with 6 values per line.
Examples:

// Example 2: 2D Matrix (2D Tensor)
{
  // Define the shape of the StorageView (12x12 matrix)
  ctranslate2::Shape shape = {12, 12};

  // Create a StorageView of type FLOAT32 and initialize with some values
  ctranslate2::StorageView storage_view(shape, ctranslate2::DataType::FLOAT32);

  // Fill the StorageView with some data
  float data[144];
  for (int i = 0; i < 144; ++i) {
    data[i] = static_cast<float>(i + 1);
  }
  storage_view.copy_from(data, 144, ctranslate2::Device::CPU);

  // Output the contents and metadata of the StorageView
  std::cout << storage_view << std::endl;
}

// Example 3: 3D Tensor
{
  // Define the shape of the StorageView (9x9x9 tensor)
  ctranslate2::Shape shape = {9, 9, 9};

  // Create a StorageView of type FLOAT32 and initialize with some values
  ctranslate2::StorageView storage_view(shape, ctranslate2::DataType::FLOAT32);

  // Fill the StorageView with some data
  float data[729];
  for (int i = 0; i < 729; ++i) {
    data[i] = static_cast<float>(i + 1);
  }
  storage_view.copy_from(data, 729, ctranslate2::Device::CPU);

  // Output the contents and metadata of the StorageView
  std::cout << storage_view << std::endl;
}

Data (2D Matrix):
[[1, 2, 3, ..., 10, 11, 12], 
[13, 14, 15, ..., 22, 23, 24], 
[25, 26, 27, ..., 34, 35, 36], 
..., 
[109, 110, 111, ..., 118, 119, 120], 
[121, 122, 123, ..., 130, 131, 132], 
[133, 134, 135, ..., 142, 143, 144]]
[device:cpu:0, dtype:float32, storage viewed as 12x12]

Data (3D Tensor):
[[[1, 2, 3, ..., 7, 8, 9], 
[10, 11, 12, ..., 16, 17, 18], 
[19, 20, 21, ..., 25, 26, 27], 
..., 
[55, 56, 57, ..., 61, 62, 63], 
[64, 65, 66, ..., 70, 71, 72], 
[73, 74, 75, ..., 79, 80, 81]], 
[[10, 11, 12, ..., 16, 17, 18], 
[19, 20, 21, ..., 25, 26, 27], 
[28, 29, 30, ..., 34, 35, 36], 
..., 
[64, 65, 66, ..., 70, 71, 72], 
[73, 74, 75, ..., 79, 80, 81], 
[82, 83, 84, ..., 88, 89, 90]], 
[[19, 20, 21, ..., 25, 26, 27], 
[28, 29, 30, ..., 34, 35, 36], 
[37, 38, 39, ..., 43, 44, 45], 
..., 
[73, 74, 75, ..., 79, 80, 81], 
[82, 83, 84, ..., 88, 89, 90], 
[91, 92, 93, ..., 97, 98, 99]], 
..., 
[[55, 56, 57, ..., 61, 62, 63], 
[64, 65, 66, ..., 70, 71, 72], 
[73, 74, 75, ..., 79, 80, 81], 
..., 
[109, 110, 111, ..., 115, 116, 117], 
[118, 119, 120, ..., 124, 125, 126], 
[127, 128, 129, ..., 133, 134, 135]], 
[[64, 65, 66, ..., 70, 71, 72], 
[73, 74, 75, ..., 79, 80, 81], 
[82, 83, 84, ..., 88, 89, 90], 
..., 
[118, 119, 120, ..., 124, 125, 126], 
[127, 128, 129, ..., 133, 134, 135], 
[136, 137, 138, ..., 142, 143, 144]], 
[[73, 74, 75, ..., 79, 80, 81], 
[82, 83, 84, ..., 88, 89, 90], 
[91, 92, 93, ..., 97, 98, 99], 
..., 
[127, 128, 129, ..., 133, 134, 135], 
[136, 137, 138, ..., 142, 143, 144], 
[145, 146, 147, ..., 151, 152, 153]]]
[device:cpu:0, dtype:float32, storage viewed as 9x9x9]

include/ctranslate2/storage_view.h Outdated Show resolved Hide resolved
include/ctranslate2/storage_view.h Outdated Show resolved Hide resolved
src/storage_view.cc Outdated Show resolved Hide resolved
src/storage_view.cc Outdated Show resolved Hide resolved
@tirivo tirivo force-pushed the dev/improve_log_message_storage_view branch from b3c01d5 to 72263c8 Compare June 4, 2024 12:58
@tirivo tirivo force-pushed the dev/improve_log_message_storage_view branch from 72263c8 to 1e4a638 Compare June 24, 2024 08:10
@minhthuc2502
Copy link
Collaborator

Please fix the pipeline

@tirivo tirivo force-pushed the dev/improve_log_message_storage_view branch 4 times, most recently from 528b282 to 952e3f5 Compare June 28, 2024 15:16
@tirivo tirivo force-pushed the dev/improve_log_message_storage_view branch from 952e3f5 to de683b7 Compare October 14, 2024 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants