sg_turs: fix missing CDB on several codepaths; sg_inq+sg_vpd: show de…

…population time in hours and minutes; sg_get_elem_status: various improvements; sg_requests: improve buffer truncation

git-svn-id: https://svn.bingwo.ca/repos/sg3_utils/trunk@1053 6180dd3e-e324-4e3e-922d-17de1ae2f315

ChangeLog

@@ -2,13 +2,18 @@ Each utility has its own version number, date of last change and
 some description at the top of its ".c" file. All utilities in the main
 directory have their own "man" pages. There is also a sg3_utils man page.
 
-Changelog for pre-release sg3_utils-1.49 [20230928] [svn: r1051]
+Changelog for pre-release sg3_utils-1.49 [20231015] [svn: r1053]
   - apply https://github.com/doug-gilbert/sg3_utils/pull/39
     and its revision [20230807] mainly for Android
   - sg_inq: update version descriptors to T10 table 20230814
   - sg_wr_mode: fix --contents and --cfile= handling
   - sg_vpd: inform about SPC-6 (sg_inq knew already)
     https://github.com/doug-gilbert/sg3_utils/issues/40
+  - sg_inq+sg_vpd: show depopulation time in hours and
+    minutes as well as its raw form: seconds
+  - sg_turs: fix missing CDB on several codepaths
+  - sg_requests: improve buffer truncation
+  - sg_get_elem_status: various improvements
   - lib/sg_pt_solaris.c : fix compilation issue per
     https://github.com/doug-gilbert/sg3_utils/pull/41
     - add --mfile=MF option for decoding mask in the MF file

debian/changelog

@@ -2,7 +2,7 @@ sg3-utils (1.49-0.1) unstable; urgency=low
 
   * New upstream version
 
- -- Douglas Gilbert <[email protected]>  Thu, 14 Sep 2023 21:00:00 -0400
+ -- Douglas Gilbert <[email protected]>  Sun, 01 Oct 2023 13:00:00 -0400
 
 sg3-utils (1.48-0.1) unstable; urgency=low
 

doc/sg3_utils.8

@@ -1,4 +1,4 @@
-.TH SG3_UTILS "8" "September 2023" "sg3_utils\-1.49" SG3_UTILS
+.TH SG3_UTILS "8" "October 2023" "sg3_utils\-1.49" SG3_UTILS
 .SH NAME
 sg3_utils \- a package of utilities for sending SCSI commands
 .SH SYNOPSIS

doc/sg3_utils_json.8

@@ -1,4 +1,4 @@
-.TH SG3_UTILS_JSON "8" "September 2023" "sg3_utils\-1.49" SG3_UTILS
+.TH SG3_UTILS_JSON "8" "October 2023" "sg3_utils\-1.49" SG3_UTILS
 .SH NAME
 sg3_utils_json \- JSON output for some sg3_utils utilities
 .SH SYNOPSIS

doc/sg_get_elem_status.8

@@ -37,6 +37,11 @@ Arguments to long options are mandatory for short options as well.
 .TP
 \fB\-b\fR, \fB\-\-brief\fR
 when used once, the output of each physical element status descriptor is
+abbreviated compared to when this option is not given. Also the
+term "associated capacity: not specified;" is dropped as it doesn't convey
+much information.
+.br
+When used twice, the output of each physical element status descriptor is
 reduced to: <element_id>: <element_type>,<element_health> . All three are
 output as decimal integers. When used twice the "Element descriptors:"
 line introducing the status descriptors is not output. When used three

doc/sg_inq.8

@@ -25,7 +25,7 @@ This utility, when \fIDEVICE\fR is given, sends a SCSI INQUIRY command to it
 then outputs the response. All SCSI devices are meant to respond to
 a "standard" INQUIRY command with at least a 36 byte response (in SCSI 2 and
 higher). An INQUIRY is termed as "standard" when both the EVPD and CmdDt (now
-obsolete) bits are clear. Formally (i.e. as per SPC stardards) the name of
+obsolete) bits are clear. Formally (i.e. as per SPC standards) the name of
 a standard INQUIRY response is the "standard INQUIRY data format" but here
 the "standard INQUIRY response" is used as it is shorter and more descriptive.
 .PP

doc/sg_rem_rest_elem.8

@@ -1,4 +1,4 @@
-.TH SG_REM_REST_ELEM "8" "May 2023" "sg3_utils\-1.48" SG3_UTILS
+.TH SG_REM_REST_ELEM "8" "October 2023" "sg3_utils\-1.49" SG3_UTILS
 .SH NAME
 sg_rem_rest_elem \- send SCSI remove or restore element command
 .SH SYNOPSIS
@@ -23,7 +23,7 @@ where only a specified number of commands will be executed (e.g. INQUIRY and
 REPORT LUNS) until the operation is complete. Other commands will receive
 sense data with a sense key of NOT READY and an additional sense code
 of 'Depopulation in progress' (for RMEAT) or 'Depopulation restoration in
-progress' (for RSEAR).
+progress' (for RSEAR). See the NOTES section below for more information.
 .PP
 The REMOVE ELEMENT AND TRUNCATE has a close relative in ZBC\-2 called the
 REMOVE ELEMENT AND MODIFY ZONES [RMEMZ] command. See the sg_zone utility
@@ -84,6 +84,47 @@ set (see sg_get_elem_status) are candidates for future restoration.
 A (storage) element of a rotating hard disk is one side of a platter
 typically associated with one head. Such hard disks typically have multiple
 platters with two heads per platter (i.e. one head each side of the platter).
+.PP
+Only one element can be depopulated at a time. It seems that user data is
+striped across various platters (manufacturers pointedly don't describe
+how that is done) so depopulation can take a very long time irrespective
+of which element (e.g. the last element) is depopulated. The DEPOPULATION
+TIME field in the Block Device Characteristics VPD page gives an indication
+of how long the depopulation of one element will take. The unit of that
+field is seconds and for reference, there are 86,400 seconds in a day.
+Once a \fI\-\-remove\fR or \fI\-\-restore\fR based command has successfully
+returned, the disk goes into what can be considered as "depopulation" mode.
+While a disk is in "depopulation" mode, all media access commands (and some
+others) are rejected with a NOT READY sense code and additional sense code
+of 0x4,0x24 or 0x4,0x25 (the latter one is for "depopulation restoration
+in progress").
+.PP
+While this facility is described by reference to hard (spinning) disks, the
+commands are defined in general terms so that they might also apply to solid
+state disks (SSDs) that are implemented using multiple ICs (die_s).
+Depopulation may be useful if one of those ICs becomes faulty.
+.PP
+At time of writing, OS and HBA support for depopulating hard disks is
+incomplete. For example Linux as of lk 6.5.7 does not properly handle the
+situation when a hard disk is in "depopulation" mode. Also at least one
+SAS-4 HBA has trouble booting when one of its attached hard disks is
+in "depopulation" mode. Suggestion: depopulating a head on a hard
+disk (or restoring it) should be done on a non-production system that
+is not used for any other purpose during the depopulation.
+.PP
+When a hard disk is in "depopulation" mode a REQUEST SENSE command
+can be sent to obtain "progress" information. This can be done with
+the 'sg_requests' or 'sg_requests --progress' utility.
+.PP
+In summary, "head" depopulation on a hard disk can be viewed as a type
+of format operation that takes a failed or failing disk with media
+errors all on one side of a platter, and produces a properly functioning
+disk with (slightly) reduced capacity. Finally the T10 specifications
+do not say that depopulation on a disk will lead to
+.B all
+data will be lost, but it is a safe assumption that all data will indeed be
+lost. So the normal warnings about taking backups before attempting
+depopulation apply.
 .SH EXIT STATUS
 The exit status of sg_rem_rest_elem is 0 when it is successful. Otherwise see
 the sg3_utils(8) man page.
@@ -97,4 +138,4 @@ Copyright \(co 2022\-2023 Douglas Gilbert
 This software is distributed under a BSD\-2\-Clause license. There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 .SH "SEE ALSO"
-.B sg_get_elem_status,sg_zone(sg3_utils)
+.B sg_get_elem_status,sg_requests,sg_zone(sg3_utils)

lib/sg_lib.c

@@ -166,8 +166,12 @@ sg_get_command_str(const uint8_t * cdbp, int sz, bool cmd_name, int blen,
 {
     int k, j, jj;
 
-    if ((cdbp == NULL) || (b == NULL) || (blen < 1))
+    if ((b == NULL) || (blen < 1))
         return b;
+    else if ((cdbp == NULL) || (sz < 1)) {
+        snprintf(b, blen, "<empty>");
+        return b;
+    }
     if (cmd_name && (blen > 16)) {
         sg_get_command_name(cdbp, 0, blen, b);
         j = (int)strlen(b);
@@ -360,7 +364,7 @@ sg_get_additional_sense_str(int asc, int ascq, bool add_sense_leadin,
     }
     for (k = 0; sg_lib_asc_ascq_range[k].text; ++k) {
         const struct sg_lib_asc_ascq_range_t * ei2p =
-					&sg_lib_asc_ascq_range[k];
+                                        &sg_lib_asc_ascq_range[k];
 
         if ((ei2p->asc == asc) &&
             (ascq >= ei2p->ascq_min)  &&
@@ -1524,18 +1528,13 @@ sg_get_sense_descriptors_str(const char * lip, const uint8_t * sbp,
     uint16_t sct_sc;
     bool processed;
     const uint8_t * descp;
-    char z[64];
     static const char * dtsp = "   >> descriptor too short";
     static const char * eccp = "Extended copy command";
     static const char * ddp = "destination device";
 
     if ((NULL == b) || (blen <= 0))
         return 0;
     b[0] = '\0';
-    if (lip)
-        sg_scnpr(z, sizeof(z), "%.60s  ", lip);
-    else
-        sg_scnpr(z, sizeof(z), "  ");
     if ((sb_len < 8) || (0 == (add_sb_len = sbp[7])))
         return 0;
     add_sb_len = (add_sb_len < (sb_len - 8)) ? add_sb_len : (sb_len - 8);
@@ -1767,9 +1766,16 @@ sg_get_sense_descriptors_str(const char * lip, const uint8_t * sbp,
             else
                 n += sg_scn3pr(b, blen, n, "%s    Usage reason: "
                                "reserved[%d]\n", lip, descp[3]);
-            n += sg_get_designation_descriptor_str(z, descp + 4, descp[1] - 2,
-                                                   true, false, blen - n,
-                                                   b + n);
+            {
+                char z[96];
+
+                if (lip)
+                    sg_scnpr(z, sizeof(z), "%.60s  ", lip);
+                else
+                    sg_scnpr(z, sizeof(z), "  ");
+                n += sg_get_designation_descriptor_str(z, descp + 4,
+                               descp[1] - 2, true, false, blen - n, b + n);
+            }
             break;
         case 0xf:       /* Added in SPC-5 rev 10 (for Write buffer) */
             n += sg_scn3pr(b, blen, n, "Microcode activation ");
@@ -1872,21 +1878,21 @@ sg_get_sense_str(const char * lip, const uint8_t * sbp, int sb_len,
     bool descriptor_format = false;
     bool sdat_ovfl = false;
     bool valid_info_fld;
-    int len, progress, n, r, pr, rem, blen;
+    int len, progress, n, r, pr, rem;
     unsigned int info;
     uint8_t resp_code;
     const char * ebp = NULL;
     char ebuff[64];
     char b[256];
     struct sg_scsi_sense_hdr ssh;
+    static const int blen = sizeof(b);
 
     if ((NULL == cbp) || (cblen <= 0))
         return 0;
     else if (1 == cblen) {
         cbp[0] = '\0';
         return 0;
     }
-    blen = sizeof(b);
     if (NULL == lip)
         lip = "";
     if ((NULL == sbp) || (sb_len < 1))

lib/sg_lib_data.c

@@ -19,8 +19,8 @@
 #include "sg_lib_data.h"
 
 
-const char * const sg_lib_version_str = "3.08 20230623";
-/* spc6r08, sbc5r04, zbc2r13 */
+const char * const sg_lib_version_str = "3.09 20231014";
+/* spc6r10, sbc5r04, zbc2r13 */
 
 
 /* indexed by pdt; those that map to own index do not decay */

lib/sg_pr2serr.c

@@ -101,5 +101,10 @@ sg_scn3pr(char * fcp, int fcp_len, int off, const char * fmt, ...)
     va_start(args, fmt);
     n = vsnprintf(fcp + off, fcp_len - off, fmt, args);
     va_end(args);
+#ifdef DEBUG
+    if (n >= cp_max_len)        /* make noise when truncation */
+        pr2ws("%s: truncated [n=%d]: %s; 'fmt' string: %s\n", __func__, n,
+              fcp, fmt);
+#endif
     return (n < cp_max_len) ? n : (cp_max_len - 1);
 }

sg3_utils.spec

@@ -84,7 +84,7 @@ fi
 %{_libdir}/*.a
 
 %changelog
-* Thu Sep 14 2023 - dgilbert at interlog dot com
+* Sun Oct 01 2023 - dgilbert at interlog dot com
 - track t10 changes
   * sg3_utils-1.49
 

src/sg_get_elem_status.c

@@ -38,7 +38,7 @@
  * given SCSI device.
  */
 
-static const char * version_str = "1.23 20230619";      /* sbc5r04 */
+static const char * version_str = "1.24 20231015";      /* sbc5r04 */
 
 #define MY_NAME "sg_get_elem_status"
 
@@ -120,7 +120,8 @@ usage()
             "[--verbose]\n"
             "                           [--version] DEVICE\n"
             "  where:\n"
-            "    --brief|-b        one descriptor per line\n"
+            "    --brief|-b        reduce amount of output, can be used "
+            "several times\n"
             "    --filter=FLT|-f FLT    FLT is 0 (def) for all physical "
             "elements;\n"
             "                           1 for out of spec and depopulated "
@@ -259,31 +260,47 @@ decode_elem_status_desc(const uint8_t * bp, struct gpes_desc_t * pedp)
 }
 
 static bool
-fetch_health_str(uint8_t health, char * bp, int max_blen)
+fetch_health_str(uint8_t health, bool short_str, char * bp, int max_blen)
 {
     bool add_val = false;
     const char * cp = NULL;
 
     if  (0 == health)
         cp = "not reported";
     else if (health < 0x64) {
-        cp = "within manufacturer's specification limits";
+        if (short_str)
+            cp = "within mfr's specs";
+        else
+            cp = "within manufacturer's specification limits";
+
         add_val = true;
     } else if (0x64 == health) {
-        cp = "at manufacturer's specification limits";
+        if (short_str)
+            cp = "at mfr's spec limits";
+        else
+            cp = "at manufacturer's specification limits";
         add_val = true;
     } else if (health < 0xd0) {
-        cp = "outside manufacturer's specification limits";
+        if (short_str)
+            cp = "outside mfr's spec limits";
+        else
+            cp = "outside manufacturer's specification limits";
         add_val = true;
     } else if (health < 0xfb) {
         cp = "reserved";
         add_val = true;
     } else if (0xfb == health)
-        cp = "depopulation revocation completed, errors detected";
+        if (short_str)
+            cp = "depop revoc completed, errors detected";
+        else
+            cp = "depopulation revocation completed, errors detected";
     else if (0xfc == health)
         cp = "depopulation revocation in progress";
     else if (0xfd == health)
-        cp = "depopulation completed, errors detected";
+        if (short_str)
+            cp = "depop completed, errors detected";
+        else
+            cp = "depopulation completed, errors detected";
     else if (0xfe == health)
         cp = "depopulation operations in progress";
     else if (0xff == health)
@@ -651,9 +668,9 @@ main(int argc, char * argv[])
     }
 
     sgj_haj_vi(jsp, jop, 0, "Number of descriptors",
-               SGJ_SEP_COLON_1_SPACE, num_desc, true);
+               SGJ_SEP_COLON_1_SPACE, num_desc, false);
     sgj_haj_vi(jsp, jop, 0, "Number of descriptors returned",
-               SGJ_SEP_COLON_1_SPACE, num_desc_ret, true);
+               SGJ_SEP_COLON_1_SPACE, num_desc_ret, false);
     sgj_haj_vi(jsp, jop, 0, "Identifier of element being depopulated",
                SGJ_SEP_COLON_1_SPACE, id_elem_depop, true);
     if (cur_max_num_depop > 0)
@@ -690,34 +707,45 @@ main(int argc, char * argv[])
             sgj_js_nv_istr(jsp, jo2p, "physical_element_type",
                            a_ped.phys_elem_type, "meaning", cp);
             j = a_ped.phys_elem_health;
-            fetch_health_str(j, b, blen);
+            fetch_health_str(j, false, b, blen);
             sgj_js_nv_istr(jsp, jo2p, "physical_element_health", j, NULL, b);
             sgj_js_nv_ihex(jsp, jo2p, "associated_capacity",
                            (int64_t)a_ped.assoc_cap);
             sgj_js_nv_o(jsp, jap, NULL /* name */, jo2p);
-        } else if (op->do_brief) {
+        } else if (op->do_brief > 1) {
             sgj_pr_hr(jsp, "%u: %u,%u\n", a_ped.elem_id, a_ped.phys_elem_type,
                       a_ped.phys_elem_health);
         } else {
             char b2[144];
             static const int b2len = sizeof(b2);
 
-            m = sg_scnpr(b2, b2len, "[%d] identifier: 0x%06x", k + 1,
-                         a_ped.elem_id);
-            if (sg_all_ffs((const uint8_t *)&a_ped.assoc_cap, 8))
-                m += sg_scn3pr(b2, b2len, m,
-                               "  associated LBs: not specified;  ");
+            if (op->do_brief > 0)       /* can only be 0 or 1 here */
+                m = sg_scnpr(b2, b2len, "[%d] id: 0x%x", k + 1,
+                             a_ped.elem_id);
             else
-                m += sg_scn3pr(b2, b2len, m, "  associated LBs: 0x%" PRIx64
-                               ";  ", a_ped.assoc_cap);
+                m = sg_scnpr(b2, b2len, "[%d] identifier: 0x%06x", k + 1,
+                             a_ped.elem_id);
+            if (sg_all_ffs((const uint8_t *)&a_ped.assoc_cap, 8)) {
+                if (op->do_brief > 0)
+                    m += sg_scn3pr(b2, b2len, m, "  ");
+                else
+                    m += sg_scn3pr(b2, b2len, m,
+                                   "  associated capacity: not specified;  ");
+            } else
+                m += sg_scn3pr(b2, b2len, m, "  associated capacity: 0x%"
+                               PRIx64 ";  ", a_ped.assoc_cap);
             m += sg_scn3pr(b2, b2len, m, "health: ");
             j = a_ped.phys_elem_health;
-            if (fetch_health_str(j, b, blen))
+            if (fetch_health_str(j, (op->do_brief > 0), b, blen))
                 m += sg_scn3pr(b2, b2len, m, "%s <%d>", b, j);
             else
                 m += sg_scn3pr(b2, b2len, m, "%s", b);
-            if (a_ped.restoration_allowed)
-                sg_scn3pr(b2, b2len, m, " [restoration allowed [RALWD]]");
+            if (a_ped.restoration_allowed) {
+                if (op->do_brief > 0)
+                    sg_scn3pr(b2, b2len, m, " [RALWD]");
+                else
+                    sg_scn3pr(b2, b2len, m, " [restoration allowed [RALWD]]");
+            }
             sgj_pr_hr(jsp, "%s\n", b2);
         }
     }

src/sg_inq.c

@@ -53,7 +53,7 @@
 
 #include "sg_vpd_common.h"  /* for shared VPD page processing with sg_vpd */
 
-static const char * version_str = "2.50 20230926";  /* spc6r10, sbc5r04 */
+static const char * version_str = "2.51 20231014";  /* spc6r10, sbc5r04 */
 
 #define MY_NAME "sg_inq"