package: 'never imported' yet shows a version #511

dlangille · 2023-12-11T00:58:16Z

At https://www.freshports.org/sysutils/anvil/#packages under FreeBSD:15:powerpc64 the box under the latest column contains a version (0.0.20). The mouse-hover text says:

repo not found
never imported
2023-12-11 00:00 - last checked by FreshPorts

If it has never been imported, where did the version 0.0.20 come from?

The text was updated successfully, but these errors were encountered:

dlangille · 2023-12-11T13:35:24Z

Issue is not present on dev, where it says:

dlangille · 2023-12-11T13:35:50Z

The issue is visible on stage, but not on test.

dlangille · 2023-12-11T13:38:15Z

Code is: https://github.com/FreshPorts/freshports/blob/2.8.6/classes/port-display.php#L776

dlangille · 2023-12-11T13:49:40Z

Data is:

freshports.stage=# select id, short_description from ports where element_id = (select element_id from element_pathname where pathname = '/ports/head/sysutils/anvil');
  id   |            short_description            
-------+-----------------------------------------
 44431 | Tools for distributing ssl certificates
(1 row)

freshports.stage=# select * from PortPackages(44431)
where abi = 'FreeBSD:15:powerpc64';
 package_name |         abi          | package_version_latest | package_version_quarterly | last_checked_latest | repo_date_latest | import_date_latest | processed_date_latest | last_checked_quarterly | repo_date_quarterly | import_date_quarterly | processed_date_quarterly 
--------------+----------------------+------------------------+---------------------------+---------------------+------------------+--------------------+-----------------------+------------------------+---------------------+-----------------------+--------------------------
 anvil        | FreeBSD:15:powerpc64 | 0.0.20                 |                           | 2023-12-11 13:00    |                  |                    |                       | 2023-12-11 13:00       |                     |                       | 
(1 row)

freshports.stage=#

dlangille · 2023-12-11T14:05:24Z

The output above is missing the following information.

Meaning: the repo has been imported, and import_date_latest above should be the import_date shown below.

freshports.stage=# select * from package_imports where abi_id = (select id from abi where name = 'FreeBSD:15:powerpc64');
  id  | abi_id | package_set |             date              | inserts | updates | deletes |         last_checked          |       repo_date        |          import_date          |        processed_date         
------+--------+-------------+-------------------------------+---------+---------+---------+-------------------------------+------------------------+-------------------------------+-------------------------------
 6333 |     41 | latest      | 2023-10-10 11:00:55.679798+00 |   29368 |       0 |       0 | 2023-10-10 11:00:04.149062+00 | 2023-10-09 18:52:05+00 | 2023-10-10 11:00:23.992868+00 | 2023-10-10 11:00:55.679798+00
(1 row)

dlangille · 2023-12-11T14:13:04Z

Now I'm confused. This says they have never been imported:

freshports.stage=# select * from packages_last_checked where abi_id = (select id from abi where name = 'FreeBSD:15:powerpc64');
 abi_id | package_set |         last_checked          | repo_date | import_date | processed_date 
--------+-------------+-------------------------------+-----------+-------------+----------------
     41 | latest      | 2023-12-11 14:00:04.126578+00 |           |             | 
     41 | quarterly   | 2023-12-11 14:00:04.126578+00 |           |             | 
(2 rows)

freshports.stage=#

Logs confirm it has not been imported (today).

dlangille · 2023-12-11T14:27:07Z

It shows:

freshports.stage=# select * from packages_raw where package_origin = 'sysutils/anvil' and abi = 'FreeBSD:15:powerpc64';
    id     | package_origin | package_name | package_version |         abi          | abi_id | port_id | package_set 
-----------+----------------+--------------+-----------------+----------------------+--------+---------+-------------
 185277569 | sysutils/anvil | anvil        | 0.0.20          | FreeBSD:15:powerpc64 |     41 |   44431 | latest
(1 row)

Yet, https://pkg.freebsd.org/FreeBSD:15:powerpc64/latest is a 404

dlangille · 2023-12-11T14:39:38Z

Let's try this approach:

freshports.stage=# delete from packages_raw;
DELETE 1521663
freshports.stage=#

Yes, that's 1,521,663 rows...

dlangille · 2023-12-11T14:41:17Z

And start from scratch:

freshports.dev=# \d packages_last_checked
                    Table "public.packages_last_checked"
     Column     |           Type           | Collation | Nullable | Default 
----------------+--------------------------+-----------+----------+---------
 abi_id         | integer                  |           | not null | 
 package_set    | package_sets             |           |          | 
 last_checked   | timestamp with time zone |           |          | 
 repo_date      | timestamp with time zone |           |          | 
 import_date    | timestamp with time zone |           |          | 
 processed_date | timestamp with time zone |           |          | 
Foreign-key constraints:
    "packages_last_checked_abi_id_fk" FOREIGN KEY (abi_id) REFERENCES abi(id) ON UPDATE CASCADE ON DELETE CASCADE

freshports.dev=# update packages_last_checked set import_date = null, processed_date = null, last_checked=null;
UPDATE 74
freshports.dev=#

dlangille · 2023-12-11T22:43:32Z

I think this may be the lack of delete on packages_raw.

dlangille · 2023-12-11T22:43:54Z

dev, test, and stage are right.

dlangille · 2023-12-12T00:01:13Z

I tried fetching earlier and failed. The file is not there.

[0:07 dev-ingress01 dan ~/tmp/package-importing] % fetch https://pkg.freebsd.org/FreeBSD:15:powerpc64/latest/packagesite.txz 
fetch: https://pkg.freebsd.org/FreeBSD:15:powerpc64/latest/packagesite.txz: Not Found

For a successful fetch, this time FreeBSD:15:amd64, we have:

[23:53 dev-ingress01 dan ~/tmp/package-importing] % fetch https://pkg.freebsd.org/FreeBSD:15:amd64/latest/packagesite.txz
packagesite.txz                                       7179 kB   11 MBps    01s


[23:53 dev-ingress01 dan ~/tmp/package-importing] % tar -xzf packagesite.txz

[23:54 dev-ingress01 dan ~/tmp/package-importing] % ls -l
total 25654
-rw-r--r--  1 dan dan  7352044 2023.12.08 02:04 packagesite.txz
-rw-r--r--  1 dan dan 53982039 2023.12.08 02:03 packagesite.yaml
-rw-r--r--  1 dan dan      451 1970.01.01 00:00 packagesite.yaml.pub
-rw-r--r--  1 dan dan      256 1970.01.01 00:00 packagesite.yaml.sig

The date shown at https://pkg.freebsd.org/FreeBSD:15:amd64/ (2023-Oct-10 20:06) does not match the date on the file (2023.12.08 02:04). FreshPorts always uses the date shown above on packagesite.txz (or it might be the date on packagesite.yaml - I'm not sure now.

Looking at https://www.freshports.org/sysutils/anvil/#packages for FreeBSD:15:amd64 I find:

Reproduced below via copy/paste:

2023-12-08 02:04 - repo build date
2023-12-09 13:00 - processed by FreshPorts
2023-12-11 21:00 - last checked by FreshPorts

In this case, it's clear that it took some time for packagesite.txz to be made public. FreshPorts checks every hour at the top of the hour.

markmi · 2023-12-12T00:06:47Z

If you saw my earlier note that I later deleted, I had confused

latest/
with:
latest/All/

It is latest/All/ that is restricted.

So a 404 for latest/ is significant about if there is data available or not currently.

In fact, https://pkg.freebsd.org/FreeBSD:13:powerpc64/quarterly/All/ gives 403 (not 404).

dlangille · 2023-12-12T00:18:43Z

I did, and it was very useful. See my reply above. It helped bring out some detail which might be useful for future readers. I hope that the issues help future maintainers understand how things works.

dlangille · 2023-12-12T00:28:50Z

I think this may be the lack of delete on packages_raw.

Options for this delete:

import-via-copy-packagesite-all-raw-fields.py - before the copy
import_packagesite.py- oh wait, there's a delete in there - perhaps it is faulty.

Said function is incredibly simple:

CREATE OR REPLACE FUNCTION PackagesRawDeleteForABIPackageSet( a_abi_name text, a_package_set package_sets) RETURNS void AS $$
   DELETE FROM packages_raw
    WHERE abi         = a_abi_name
      AND package_set = a_package_set;
$$ LANGUAGE SQL VOLATILE;

There's a point - the data is deleted before import. If an import for a given ABI never occurs again, that data remains forever.

Perhaps the delete for packages_raw needs to occur also after a successful population of the packages table.

Perhaps within the UpdatePackagesFromRawPackages PostgreSQL function (re #449).

markmi · 2023-12-12T00:29:35Z

I'll note that I'm confused about your comparing amd64 dates/times with powerpc64 dates/times. I would not expect a match. As seen here, using FreeBSD:13 materials where both are accessible. . .

https://pkg.freebsd.org/FreeBSD:13:powerpc64/quarterly/ has packagesite.pkg date/time 2023-Nov-22 11:38 showing.
https://pkg.freebsd.org/FreeBSD:13:amd64/quarterly/ has packagesite.pkg date/time 2023-Dec-05 12:14 showing.

So: a fair amount of time between the two.

dlangille · 2023-12-12T00:48:20Z

I'm going to start with adding this:

SELECT PackagesRawDeleteForABIPackageSet( a_abi_name, a_package_set);

Just before the RETURN QUERY shown at the end of this function.

Then, I'll delete everything from packages_raw and see how that goes.

CREATE OR REPLACE FUNCTION UpdatePackagesFromRawPackages( IN a_abi_name text, IN a_package_set package_sets)

-- this function takes data from the Raw Packages table and populates the
-- packages table.
--
-- See https://github.com/FreshPorts/freshports/issues/449
--
    RETURNS TABLE(inserts integer, updates integer, deletes integer)
    LANGUAGE plpgsql VOLATILE
    AS $$
  DECLARE
    l_abi_id  integer;
    l_inserts integer;
    l_deletes integer;
    l_updates integer;
  BEGIN
    SELECT id
      INTO l_abi_id
      FROM abi
     WHERE name = a_abi_name;

    IF NOT FOUND THEN
        RAISE EXCEPTION 'cannot find id for ABI %', a_abi_name;
    END IF;

    -- set abi_id and port_id before updating the packages table.
    UPDATE packages_raw PR
       SET abi_id  = l_abi_id,
           port_id = P.id
      FROM ports P, element_pathname EP
     WHERE PR.abi         = a_abi_name
       AND PR.package_set = a_package_set
       AND EP.pathname    = '/ports/head/' || PR.package_origin
       AND P.element_id   = EP.element_id;

    -- this is required to allow the DELETE to properly plan the operation
    analyze packages_raw;

    UPDATE packages p
       SET package_version = PR.package_version
      FROM packages_raw PR
     WHERE P.abi_id           = PR.abi_id
       AND P.package_set      = PR.package_set
       AND P.package_name     = PR.package_name
       AND P.port_id          = PR.port_id
       AND P.package_version != PR.package_version
       AND PR.abi             = a_abi_name
       AND PR.package_set     = a_package_set;

    GET DIAGNOSTICS l_updates = ROW_COUNT;

    -- populate the packages table.
    -- this should be rather snappy now that all the values are set.
    --
    -- 2023-09-30 - See that WHERE NOT EXISTS? It seems like I was trying to
    -- avoid overwriting existing data. I now believe that's the wrong approach
    -- The data in pacakges_raw reflects what is now available from the
    -- packages server. It completely supercedes any existing information for
    -- this ABI
    -- 
    -- some background: https://news.freshports.org/2020/04/17/1325/
    -- 
    INSERT INTO packages (abi_id, port_id, package_version, package_name, package_set)
    SELECT abi_id, port_id, package_version, package_name, package_set
      FROM packages_raw PR
     WHERE NOT EXISTS (
     SELECT *
       FROM packages P
      WHERE P.abi_id           = PR.abi_id
        AND P.package_set      = PR.package_set
        AND P.package_name     = PR.package_name
        AND P.port_id          = PR.port_id
        AND P.package_version  = PR.package_version
        AND PR.abi             = a_abi_name
        AND PR.package_set     = a_package_set)
     AND PR.abi_id  IS NOT NULL
     AND PR.port_id IS NOT NULL
     AND PR.abi             = a_abi_name
     AND PR.package_set     = a_package_set;

    GET DIAGNOSTICS l_inserts = ROW_COUNT;


    DELETE FROM packages P
     WHERE P.abi_id      = l_abi_id
       AND P.package_set = a_package_set
       AND NOT EXISTS (
         SELECT *
           FROM packages_raw PR
          WHERE P.abi_id       = PR.abi_id
            AND P.package_set  = PR.package_set
            AND P.package_name = PR.package_name
            AND P.port_id      = PR.port_id
            AND PR.abi         = a_abi_name
            AND PR.package_set = a_package_set);

    GET DIAGNOSTICS l_deletes = ROW_COUNT;

    INSERT INTO package_imports (abi_id, package_set, date, inserts, updates, deletes, last_checked, repo_date, import_date, processed_date)
    SELECT l_abi_id, a_package_set, CURRENT_TIMESTAMP, l_inserts, l_updates, l_deletes,
           PLC.last_checked, PLC.repo_date, PLC.import_date, CURRENT_TIMESTAMP
      FROM packages_last_checked PLC
     WHERE PLC.abi_id      = l_abi_id
       AND PLC.package_set = a_package_set;

    RETURN QUERY (SELECT l_inserts, l_updates, l_deletes);

  END;
$$;

dlangille · 2023-12-12T00:51:07Z

I'll note that I'm confused about comparing amd64 dates/times with powerpc64 dates/times. I would not expect a match. As seen here, using FreeBSD:13 materials where both are accessible. . .

I agree. A match is not expected. I think that came from the delete comment where the timestamp was referenced. I was showing that the time on packagesite.pkg and on https://pkg.freebsd.org/ABI/ do not correspond.

https://pkg.freebsd.org/FreeBSD:13:powerpc64/quarterly/ has packagesite.pkg date/time 2023-Nov-22 11:38 showing. https://pkg.freebsd.org/FreeBSD:13:amd64/quarterly/ has packagesite.pkg date/time 2023-Dec-05 12:14 showing.

So: a fair amount of time between the two.

Agreed. Not relevant.

dlangille · 2023-12-12T00:53:25Z

BTW, are you getting emails on each of my edits? :/

dlangille · 2023-12-12T00:57:49Z

Now in place:

[0:56 pg03 dan ~/src/freshports/database-schema] % svn di
Index: sp.txt
===================================================================
--- sp.txt	(revision 5941)
+++ sp.txt	(working copy)
@@ -3909,6 +3909,13 @@
      WHERE PLC.abi_id      = l_abi_id
        AND PLC.package_set = a_package_set;
 
+    --
+    -- delete what we just imported - let's not leave it hanging around
+    -- because a future import of this ABI may never occur
+    -- re https://github.com/FreshPorts/freshports/issues/511
+    --
+    SELECT PackagesRawDeleteForABIPackageSet( a_abi_name, a_package_set);
+
     RETURN QUERY (SELECT l_inserts, l_updates, l_deletes);
 
   END;
[0:56 pg03 dan ~/src/freshports/database-schema] %

Followed by:

[0:54 pg03 dan ~/src/freshports/database-schema] % psql freshports.dev
psql (16.1)
Type "help" for help.

freshports.dev=# begin;
BEGIN
freshports.dev=*# \i ri.txt
...
freshports.dev=*# \i sp.txt
...
freshports.dev=*# delete from packages_raw;
DELETE 1377375
freshports.dev=*# commit;
COMMIT

Now, we wait.

markmi · 2023-12-12T01:00:47Z

BTW, are you getting emails on each of my edits? :/

Yes.

markmi · 2023-12-12T01:08:22Z

BTW, are you getting emails on each of my edits? :/

Yes.

I think I misinterpreted the reference: I'm not getting multiple updates of the same message. So I think the answer you were looking for is: No.

dlangille · 2023-12-12T01:33:09Z

BTW, are you getting emails on each of my edits? :/

Yes.

I think I misinterpreted the reference: I'm not getting multiple updates of the same message. So I think the answer you were looking for is: No.

Some posts, like #511 (comment) have had 5 or 6 edits, as I explore, find, and document.

markmi · 2023-12-12T01:49:02Z

Some posts, like #511 (comment) have had 5 or 6 edits, as I explore, find, and document.

I have 8 mails currently for 511:

They are all from after my deleted message about latest/ where I'd confused things.

dlangille · 2023-12-12T15:01:30Z

Now in place:

[0:56 pg03 dan ~/src/freshports/database-schema] % svn di
Index: sp.txt
===================================================================
--- sp.txt	(revision 5941)
+++ sp.txt	(working copy)
@@ -3909,6 +3909,13 @@
      WHERE PLC.abi_id      = l_abi_id
        AND PLC.package_set = a_package_set;
 
+    --
+    -- delete what we just imported - let's not leave it hanging around
+    -- because a future import of this ABI may never occur
+    -- re https://github.com/FreshPorts/freshports/issues/511
+    --
+    SELECT PackagesRawDeleteForABIPackageSet( a_abi_name, a_package_set);
+
     RETURN QUERY (SELECT l_inserts, l_updates, l_deletes);
 
   END;
[0:56 pg03 dan ~/src/freshports/database-schema] %

Followed by:

[0:54 pg03 dan ~/src/freshports/database-schema] % psql freshports.dev
psql (16.1)
Type "help" for help.

freshports.dev=# begin;
BEGIN
freshports.dev=*# \i ri.txt
...
freshports.dev=*# \i sp.txt
...
freshports.dev=*# delete from packages_raw;
DELETE 1377375
freshports.dev=*# commit;
COMMIT

Now, we wait.

And we have

 *** /var/log/freshports//freshports-daemon.log ***
Dec 12 14:58:48 dev-ingress01 freshports[51188]: Traceback (most recent call last):
Dec 12 14:58:48 dev-ingress01 freshports[51188]:   File "/usr/local/libexec/freshports/UpdatePackagesFromRawPackages.py", line 47, in <module>
Dec 12 14:58:48 dev-ingress01 freshports[51188]:     cursUpdate.callproc('UpdatePackagesFromRawPackages', (row['abi_name'],row['package_set']))
Dec 12 14:58:48 dev-ingress01 freshports[51188]:   File "/usr/local/lib/python3.9/site-packages/psycopg2/extras.py", line 151, in callproc
Dec 12 14:58:48 dev-ingress01 freshports[51188]:     return super().callproc(procname, vars)
Dec 12 14:58:48 dev-ingress01 freshports[51188]: psycopg2.errors.SyntaxError: query has no destination for result data
Dec 12 14:58:48 dev-ingress01 freshports[51188]: HINT:  If you want to discard the results of a SELECT, use PERFORM instead.
Dec 12 14:58:48 dev-ingress01 freshports[51188]: CONTEXT:  PL/pgSQL function updatepackagesfromrawpackages(text,package_sets) line 102 at SQL statement
Dec 12 14:58:48 dev-ingress01 freshports[51188]:

SELECT PackagesRawDeleteForABIPackageSet( a_abi_name, a_package_set);

Will become:

PERFORM PackagesRawDeleteForABIPackageSet( a_abi_name, a_package_set);

dlangille · 2023-12-12T15:08:48Z

Before:

freshports.dev=# select count(*) from packages_raw;
 count 
-------
 99740
(1 row)

After the latest import:

freshports.dev=# select count(*) from packages_raw;
 count 
-------
     0
(1 row)

freshports.dev=#

dlangille · 2023-12-12T22:49:20Z

@markmi https://dev.freshports.org/ should have a clean & full import of packages. There should be no more "shows a version but never imported" situations. If you see nothing untoward, I'll save the changes and migrate towards prod.

dlangille · 2023-12-12T23:26:13Z

These changes are now on test and stage. They should reload all ABI at the top of the hour (0000 UTC).

delete from packages;
delete from packages_raw;
update packages_last_checked set import_date = null, processed_date = null, last_checked=null, repo_date = null;

markmi · 2023-12-12T23:28:29Z

https://dev.freshports.org/devel/llvm17 does have some "-" versions that have the hover text showing the likes of the following with some filled in date/times.

FreeBSD:13:armv6 llvm17 latest example (llvm17-lite too):
2021-01-25 12:43 - repo build dat
2023-12-12 22:04 - processed by FreshPorts
2023-12-12 23:00 - last checked by FreshPorts

Others, such as FreeBSD:12:aarch64 llvm17 (and llvm17-lite) latest have the likes of:
2018-10-11 05:47 - repo build date
never imported
2023-12-12 23:00 - last check by FreshPorts

devel/llvm17 was first committed on 2023-07-31, a bootstrap from devel/llvm16 .

I'll note that devel/llvm16 had "lite" added as a flavor in a commit on 2023-04-02 . llvm15 too. (Not llvm14 and before.) So lite dates before that are odd even if the date is okay for non-lite.

markmi · 2023-12-12T23:31:44Z

So far all the ones with version numbers (instead of "-") have seemed reasonable.

markmi · 2023-12-12T23:39:41Z

Another form of oddity for "-" is shown by FreeBSD:14:powerpc64le rows: The hover text for "-" (latest) is a duplicate of the hover text for the column with the 17.0.1 version number (quarterly).

I'll note that for the previously indicated FreeBSD:12:aarch64 example, the hover-text for "-" was not a duplicate of the hover-text for 17.0.1 . So: variable results in such cases.

dlangille · 2023-12-12T23:43:17Z

Others, such as FreeBSD:12:aarch64 llvm17 (and llvm17-lite) latest have the likes of: 2018-10-11 05:47 - repo build date never imported 2023-12-12 23:00 - last check by FreshPorts

For that:

[23:41 dev-ingress01 dan ~/tmp/package-importing] % fetch https://pkg.freebsd.org/FreeBSD:12:aarch64/latest/packagesite.txz
packagesite.txz                                       5944 kB   11 MBps    00s
[23:41 dev-ingress01 dan ~/tmp/package-importing] % tar -xzf packagesite.txz                                               
[23:41 dev-ingress01 dan ~/tmp/package-importing] % grep llvm17 packagesite.yaml
[23:41 dev-ingress01 dan ~/tmp/package-importing] %

So nothing in there for that. However, why is is different from FreeBSD:13:armv6 which has a value for import date?

There are values there:

freshports.dev=# select abi.name, P.* from packages_last_checked P join abi on P.abi_id = abi.id and name in ('FreeBSD:13:armv6', 'FreeBSD:12:aarch64 ');
       name       | abi_id | package_set |         last_checked          |       repo_date        |          import_date          |        processed_date         
------------------+--------+-------------+-------------------------------+------------------------+-------------------------------+-------------------------------
 FreeBSD:13:armv6 |     20 | latest      | 2023-12-13 00:00:04.245672+00 | 2021-01-25 12:43:34+00 | 2023-12-12 22:01:11.491113+00 | 2023-12-12 22:04:25.525043+00
 FreeBSD:13:armv6 |     20 | quarterly   | 2023-12-13 00:00:04.245672+00 | 2023-09-05 12:40:01+00 | 2023-12-12 22:01:14.659546+00 | 2023-12-12 22:04:32.271558+00
(2 rows)

freshports.dev=#

markmi · 2023-12-12T23:44:45Z

FYI: FreeBSD:13:armv6 llvm17 quarterly shows a different date for "- repo build date" compared to latest: 2023-09-05, not 2021-01-25 12:43 .

markmi · 2023-12-12T23:51:55Z

FYI, as far as types of examples go: devel/llvm17 is too recent for FreeBSD:*:powerpc and FreeBSD:*:powerpc64 to have examples. So, looking at devel/llvm16 can be used to see what they are like for quarterly that has builds for them.

markmi · 2023-12-13T00:00:19Z

Hmm, devel/llvm16 FreeBSD:14:powerpc64le [corrected the typo] is interesting: llvm16-lite has 16.0.5 listed for latest but llvm16 has "-" listed for latest. The hover-text for "-" (non-lite) is a duplicate of the hover-text for the 16.0.5 (lite).

(I do not know how to confirm/deny the actual status for the 3 powerpc* families.)

markmi · 2023-12-13T00:15:12Z

So far all the ones with version numbers (instead of "-") have seemed reasonable.

Still true: all the oddities that I've noticed are for "-" cells instead of for cells with actual version numbers.

markmi · 2023-12-13T01:04:43Z

In case my date reports are hard to follow by folks that might read this later: 2021-01-25 12:43:34+00 for FreeBSD:13:armv6 llvm17 latest predates the existence of devel/llvm17 by over 2 years. It can not be a valid repo_date for devel/llvm17 , no date prior to 2023-07-31 can be.

markmi · 2023-12-13T16:35:50Z

I found some odd version numbers for https://dev.freshports.org/devel/llvm/ . Odd in the sense of being really old.

The following list "90" (meaning 9.0 back then) . . .
FreeBSD:12:armv6 quarterly
FreeBSD:12:armv7 quarterly
FreeBSD:13:armv6 latest
FreeBSD:13:powerpc64 latest

The repo dates listed in the hover-text go back in 2020-Dec and 2021-Jan.

FYI: devel/llvm90 was "removed" in git back in 2023-Jan:

author Tobias C. Berner [email protected] 2023-01-04 11:03:44 +0000
committer Tobias C. Berner [email protected] 2023-01-14 23:14:56 +0000
commit 21778d8958f518e2179b1053d75cfb6c71405558 (patch)
. . .
Cleanup: Remove devel/llvm[789]0
* remove old devel/llvm70
* remove old devel/llvm80
* remove old devel/llvm90

* remove ports explicitely depending on either of the three

dlangille · 2023-12-23T17:43:43Z

Another form of oddity for "-" is shown by FreeBSD:14:powerpc64le rows: The hover text for "-" (latest) is a duplicate of the hover text for the column with the 17.0.1 version number (quarterly).

I'll note that for the previously indicated FreeBSD:12:aarch64 example, the hover-text for "-" was not a duplicate of the hover-text for 17.0.1 . So: variable results in such cases.

I think this is now consistent.

When there are values for repo build date, process by FreshPorts, and last checked by FreshPorts, but version is - === the package is not being built
A package might be built on quarterly, but not latest

dlangille · 2023-12-23T17:46:03Z

FYI: FreeBSD:13:armv6 llvm17 quarterly shows a different date for "- repo build date" compared to latest: 2023-09-05, not 2021-01-25 12:43 .

I'm not seeing the problem now. Can you still see it?

dlangille · 2023-12-23T17:58:46Z

FYI, as far as types of examples go: devel/llvm17 is too recent for FreeBSD::powerpc and FreeBSD::powerpc64 to have examples. So, looking at devel/llvm16 can be used to see what they are like for quarterly that has builds for them.

I think dev has it right. www does not.

https://www.freshports.org/devel/llvm17/#packages for FreeBSD:15:powerpc64 lists 17.0.1 with repo not found and never imported - clearly the old behavior and in error.

I'll come back to this via #519

dlangille · 2023-12-23T18:03:56Z

Hmm, devel/llvm16 FreeBSD:14:powerpcle is interesting: llvm16-lite has 16.0.5 listed for latest but llvm16 has "-" listed for latest. The hover-text for "-" (non-lite) is a duplicate of the hover-text for the 16.0.5 (lite).

(I do not know how to confirm/deny the actual status for the 3 powerpc* families.)

I don't see powerpcle listed at https://www.freshports.org/devel/llvm16/#packages

dlangille · 2023-12-23T18:11:36Z

In case my date reports are hard to follow by folks that might read this later: 2021-01-25 12:43:34+00 for FreeBSD:13:armv6 llvm17 latest predates the existence of devel/llvm17 by over 2 years. It can not be a valid repo_date for devel/llvm17 , no date prior to 2023-07-31 can be.

For https://www.freshports.org/devel/llvm17#packages and FreeBSD:13:armv6 I see:

There is no version listed. This is consistent. Do we agree?

dlangille · 2023-12-23T18:35:14Z

I found some odd version numbers for https://dev.freshports.org/devel/llvm/ . Odd in the sense of being really old.

The following list "90" (meaning 9.0 back then) . . . FreeBSD:12:armv6 quarterly FreeBSD:12:armv7 quarterly FreeBSD:13:armv6 latest FreeBSD:13:powerpc64 latest

The repo dates listed in the hover-text go back in 2020-Dec and 2021-Jan.

FYI: devel/llvm90 was "removed" in git back in 2023-Jan:

author Tobias C. Berner [email protected] 2023-01-04 11:03:44 +0000 committer Tobias C. Berner [email protected] 2023-01-14 23:14:56 +0000 commit 21778d8958f518e2179b1053d75cfb6c71405558 (patch) . . . Cleanup: Remove devel/llvm[789]0 * remove old devel/llvm70 * remove old devel/llvm80 * remove old devel/llvm90

remove ports explicitely depending on either of the three

Removing devel/llvm90 from git does not destroy the repo. It still exists. It still contains version 90.

But:

[18:16 dev-ingress01 dan ~/tmp/package-importing] % fetch https://pkg.freebsd.org/FreeBSD:12:armv6/latest/packagesite.txz 
packagesite.txz                                       5572 kB 8941 kBps    00s
[18:29 dev-ingress01 dan ~/tmp/package-importing] % time jq -rc '[1, .origin, .name, .version] | @tsv' < packagesite.yaml | > packagesite.csv
jq -rc '[1, .origin, .name, .version] | @tsv' < packagesite.yaml  0.83s user 0.04s system 99% cpu 0.872 total
 > packagesite.csv  0.00s user 0.01s system 0% cpu 0.871 total
[18:29 dev-ingress01 dan ~/tmp/package-importing] % grep devel/llvm packagesite.csv
1	devel/llvm70	llvm70	7.0.0
1	devel/llvm60	llvm60	6.0.1_2
1	devel/llvm50	llvm50	5.0.2_1
1	devel/llvm40	llvm40	4.0.1_10
[18:29 dev-ingress01 dan ~/tmp/package-importing] %

I don't see a 90 in there.

Checking the database, it's in there:

freshports.dev=# select * from packages where package_name = 'llvm';
   id    | abi_id | package_set | package_name | package_version | port_id 
---------+--------+-------------+--------------+-----------------+---------
 4031937 |      9 | quarterly   | llvm         | 15,1            |   16597
 4064717 |      1 | latest      | llvm         | 15,1            |   16597
 4098555 |      1 | quarterly   | llvm         | 15,1            |   16597
 4155229 |     15 | quarterly   | llvm         | 90              |   16597
 4178001 |     16 | quarterly   | llvm         | 90              |   16597
 4185659 |      8 | latest      | llvm         | 15,1            |   16597

Created #520 for this one.

dlangille · 2023-12-23T18:43:52Z

I think the original problem has been resolved.

markmi · 2023-12-23T18:55:51Z

FYI: FreeBSD:13:armv6 llvm17 quarterly shows a different date for "- repo build date" compared to latest: 2023-09-05, not 2021-01-25 12:43 .

I'm not seeing the problem now. Can you still see it?

I'm confused why my note was written as it was. Looking at https://pkg-status.freebsd.org/?all=1&type=package . . .

quarterly 132releng-armv6's most recent stopped:done: Tue, 05 Sep 2023 02:38:35 GMT
(that is a match for quarterly)

I do not find any default (a.k.a. latest) 13*releng-armv6

The only default (a.k.a. latest) examples I find are for: main-armv6
default main-armv6's most recent stopped:done: Thu, 12 Jan 2023 03:51:11 GMT
That predates the devel/llvm17 creation date of: 2023-07-31

I'm not sure where the 2021-01-25 date comes from. In other words: why https://pkg.freebsd.org/FreeBSD:13:armv6/latest/ is not from the Thu, 12 Jan 2023 03:51:11 GMT build that reached stopped:done: status.

(It makes sense why Freshports ends up with what https://pkg.freebsd.org/FreeBSD:13:armv6/latest/ has.)

markmi · 2023-12-23T19:01:52Z

Hmm, devel/llvm16 FreeBSD:14:powerpcle is interesting: llvm16-lite has 16.0.5 listed for latest but llvm16 has "-" listed for latest. The hover-text for "-" (non-lite) is a duplicate of the hover-text for the 16.0.5 (lite).
(I do not know how to confirm/deny the actual status for the 3 powerpc* families.)

I don't see powerpcle listed at https://www.freshports.org/devel/llvm16/#packages

Stupid typo on my part: "64" is missing. So: FreeBSD:14:powerpc64le

I still see the oddity on dev.freshports.org/devel/llmv16

markmi · 2023-12-23T19:07:37Z

In case my date reports are hard to follow by folks that might read this later: 2021-01-25 12:43:34+00 for FreeBSD:13:armv6 llvm17 latest predates the existence of devel/llvm17 by over 2 years. It can not be a valid repo_date for devel/llvm17 , no date prior to 2023-07-31 can be.

For https://www.freshports.org/devel/llvm17#packages and FreeBSD:13:armv6 I see:

. . .

There is no version listed. This is consistent. Do we agree?

I now understand that the repo date is not specific to the package or to the port at all and can predate the package/port existing. This might be appropriate for FAQ material or other notes for how to interpret what is displayed.

dlangille · 2023-12-23T19:18:29Z

I now understand that the repo date is not specific to the package or to the port at all and can predate the package/port existing. This might be appropriate for FAQ material or other notes for how to interpret what is displayed.

I have added:

The repo date is not specific to the package or to the port and can predate package/port creation.

dlangille · 2023-12-25T18:28:09Z

@markmi please see

Why does devel/llvm show version 90? #520

dlangille · 2023-12-25T19:14:42Z

@markmi also, please see

devel/llvm17 -FreeBSD:15:powerpc64 has old behavior #519

dlangille · 2024-01-12T18:21:25Z

See also https://lists.freebsd.org/archives/freebsd-hackers/2024-January/002766.html

dlangille · 2024-01-15T16:57:38Z

Edit: the below is confirmed (see new reply in URL above).

@markmi just checking: that email has nothing for FreshPorts to work on. That's something pkgmnt needs to consider first.

dlangille · 2024-01-15T19:12:41Z

Fixed, I hope.

This was referenced Dec 14, 2023

When meta.conf is not found, use default values to get packagesite.txz #514

Closed

Compile a list of FAQ about packages #515

Closed

dlangille mentioned this issue Dec 23, 2023

devel/llvm17 -FreeBSD:15:powerpc64 has old behavior #519

Closed

dlangille mentioned this issue Dec 23, 2023

Why does devel/llvm show version 90? #520

Closed

dlangille self-assigned this Dec 23, 2023

dlangille closed this as completed Jan 15, 2024

package: 'never imported' yet shows a version #511

package: 'never imported' yet shows a version #511

Comments

dlangille commented Dec 11, 2023 • edited Loading

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023 • edited Loading

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023 • edited Loading

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023

dlangille commented Dec 11, 2023

dlangille commented Dec 12, 2023 • edited Loading

markmi commented Dec 12, 2023 • edited Loading

dlangille commented Dec 12, 2023

dlangille commented Dec 12, 2023 • edited Loading

markmi commented Dec 12, 2023 • edited Loading

dlangille commented Dec 12, 2023 • edited Loading

dlangille commented Dec 12, 2023

dlangille commented Dec 12, 2023

dlangille commented Dec 12, 2023

markmi commented Dec 12, 2023

markmi commented Dec 12, 2023

dlangille commented Dec 12, 2023

markmi commented Dec 12, 2023 • edited Loading

dlangille commented Dec 12, 2023

dlangille commented Dec 12, 2023

dlangille commented Dec 12, 2023

dlangille commented Dec 12, 2023

markmi commented Dec 12, 2023

markmi commented Dec 12, 2023

markmi commented Dec 12, 2023

dlangille commented Dec 12, 2023 • edited Loading

markmi commented Dec 12, 2023

markmi commented Dec 12, 2023 • edited Loading

markmi commented Dec 13, 2023 • edited Loading

markmi commented Dec 13, 2023

markmi commented Dec 13, 2023

markmi commented Dec 13, 2023

dlangille commented Dec 23, 2023

dlangille commented Dec 23, 2023

dlangille commented Dec 23, 2023 • edited Loading

dlangille commented Dec 23, 2023

dlangille commented Dec 23, 2023

dlangille commented Dec 23, 2023 • edited Loading

dlangille commented Dec 23, 2023

markmi commented Dec 23, 2023 • edited Loading

markmi commented Dec 23, 2023

markmi commented Dec 23, 2023

dlangille commented Dec 23, 2023

dlangille commented Dec 25, 2023 • edited Loading

dlangille commented Dec 25, 2023 • edited Loading

dlangille commented Jan 12, 2024

dlangille commented Jan 15, 2024 • edited Loading

dlangille commented Jan 15, 2024

dlangille commented Dec 11, 2023 •

edited

Loading

dlangille commented Dec 11, 2023 •

edited

Loading

dlangille commented Dec 11, 2023 •

edited

Loading

dlangille commented Dec 12, 2023 •

edited

Loading

markmi commented Dec 12, 2023 •

edited

Loading

dlangille commented Dec 12, 2023 •

edited

Loading

markmi commented Dec 12, 2023 •

edited

Loading

dlangille commented Dec 12, 2023 •

edited

Loading

markmi commented Dec 12, 2023 •

edited

Loading

dlangille commented Dec 12, 2023 •

edited

Loading

markmi commented Dec 12, 2023 •

edited

Loading

markmi commented Dec 13, 2023 •

edited

Loading

dlangille commented Dec 23, 2023 •

edited

Loading

dlangille commented Dec 23, 2023 •

edited

Loading

markmi commented Dec 23, 2023 •

edited

Loading

dlangille commented Dec 25, 2023 •

edited

Loading

dlangille commented Dec 25, 2023 •

edited

Loading

dlangille commented Jan 15, 2024 •

edited

Loading