[HaikuDepot] Rethink prominence association and/or implement repository aliasing

[diver]

Downloaded haiku-r1beta2-hrev54154_31-x86_64-anyboot.iso from this post ​https://discuss.haiku-os.org/t/help-wanted-r1-beta2-boot-hardware-testing/9511

• Featured packages tab is empty and disabled
• Updated to hrev54154+81
• Featured packages tab is empty and disabled
• Renamed ~/config/cache/HaikuDepot
• Featured packages tab is still empty and disabled

All packages tab shows packages with icons, however without screenshots.

[apl-haiku]

Can you please re-run and grab the log with;

HaikuDepot -v trace

[apl-haiku]

Can you please also provide the contents of the file;

/boot/system/settings/package-repositories/HaikuPorts

[apl-haiku]

@diver - you have attached the wrong file for HaikuPorts. The one you want is at the path above and is a text "driver" file.

[diver]

Bah, sorry for the confusion. Here it is:

cfgversion=2

# This is the URL where the repository data can be accessed.
baseurl=https://eu.hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current

# This URL is an identifier for the repository that is consistent across mirrors
url=https://hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current

priority=1

[apl-haiku]

If you obtain the list of repositories that HDS knows about;

curl -L "https://depot.haiku-os.org/__repository/all-en.json.gz" | gzip -d | jq .

You will see that the only identifier (url) it knows about for x86_64 HaikuPorts is...

https://eu.hpkg.haiku-os.org:443/haikuports/master/x86_64/current

...but your configuration has an identifier (url) for the upcoming beta release...

https://hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current

Because these two do not correlate, it is not possible for HaikuDepot to connect the data available in HaikuDepotServer with the packages in the HaikuPorts repository.

You need to add a new repository to HaikuDepotServer. Please use the following codes;

• Repository : haikuportsr1b2
• Sources: haikuportsr1b2_x86_gcc2, haikuportsr1b2_x86_64

Fixing these codes after the fact will be difficult so please discuss any uncertainty or changes before proceeding.

If somebody in the admin team authenticates with HDS as an admin user then they will be able to go to the web GUI;

• Menu
• List package repositories
• Add repository
• ...
• View repository
• Add source
• ...
• Add mirror
• ...
• View repository
• Trigger import

Any problems relating to setup will appear in the logs after the import because it is at this point that it will connect to the remote and pull down the repo.info file as well as the HPKR data.

[waddlesplash]

That sounds good to me. APL, can you do that, if you are most familar? Upgrading this to blocker (though I expect it will be resolved shortly.)

[apl-haiku]

That is done now and both the repository sources are imported in HDS.

[diver]

Deleted ~/config/cache/HaikuDepot but the tab is still disabled.

I still see the following in the log:

am not updating data for depot [Haiku] as there is no web app repository source code available
am not updating data for depot [HaikuPorts] as there is no web app repository source code available

[apl-haiku]

If I pull down the repo.info file ​here then I can see this;

name HaikuPorts
vendor "Haiku Project"
summary "The HaikuPorts repository"
priority 1
url https://hpkg.haiku-os.org/haikuports/master/x86_64/current
architecture x86_64

Note that the url (identifier) field is wrong. Please discuss with the sys-admin team to get the url (identifier) fixed so that it matches the url (identifier) field in your file at;

/boot/system/settings/package-repositories/HaikuPorts

...which is...

https://hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current

Once you have done this, you should initiate a refresh of the HDS data using its ​api;

curl "https://depot.haiku-os.org/__repository/haikuportsr1b2/import"

Wait 10-15 mins (max) and you should see the "url" (identifier) updated in the HDS web UI and then it should work if restart HaikuDepot on your Haiku machine.

[waddlesplash]

Note that the url (identifier) field is wrong. Please discuss with the sys-admin team to get the url (identifier) fixed so that it matches the url (identifier) field in your file at;

Well, this is the same thing we did for R1/beta1; the HaikuPorts repository was shared and had the same identifiers. How did this work then? (Or did it not?)

How can we make it work without breaking HaikuPorts on "master"?

[apl-haiku]

In the file...

/boot/system/settings/package-repositories/HaikuPorts

...on diver's system there is a value url (identifier)...

https://hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current

Where did that come from? It is not the "master" identifier.

If you configure the url (identifier) of the R1B2 repo to be "master" then HaikuDepot applications will then only synchronize data from HDS from the actual master HaikuPorts repository and there will be inconsistency with version updates etc... I don't think this makes sense and I wouldn't do that.

I don't immediately recall the handling for R1B1. Do you have access to the repository on server disk -- perhaps you can check the repo.info file and/or an R1B1 build's /boot/system/settings/package-repositories/HaikuPorts file to see what the url (identifier) values were for that release.

[waddlesplash]

On the server, the r1beta2 HaikuPorts directory is quite literally a symlink to the master one; the two share HaikuPorts repositories, and so did r1beta1 and master up until recently. The URLs are different locally so that we can easily "fork" the two if we need to in the future, but we did not and we will probably not again.

Is there no way to get HDS to recognize, for now, the "r1beta2" identifier as "master", with the option to change this in the future?

[apl-haiku]

hi (on phone) - not easily or cleanly. can you soft-link everything but the repo.info file on the server?

[waddlesplash]

Yes, I think so. I'll look at it tomorrow.

[ttcoder]

In case the info requested is still needed:

Replying to apl-haiku:

I don't immediately recall the handling for R1B1. Do you have access to the repository on server disk -- perhaps you can check the repo.info file and/or an R1B1 build's /boot/system/settings/package-repositories/HaikuPorts file to see what the url (identifier) values were for that release.

FWIW I have an R1/b1 partition lying around, it's configured like thus:

~/Desktop> cd /SamBravo/
/SamBravo> cat system/settings/package-repositories/HaikuPorts
cfgversion=2

# This is the URL where the repository data can be accessed.
baseurl=https://eu.hpkg.haiku-os.org/haikuports/r1beta1/x86_64/current

# This URL is an identifier for the repository that is consistent across mirrors
url=https://hpkg.haiku-os.org/haikuports/master/x86_64/current

priority=1
/SamBravo> 

[waddlesplash]

I soft-linked everything but the repo.info file, and then edited the repo.info to specify "r1beta2" in its identifier, but this did not help, HaikuDepot still had an empty & grayed out featured section. Which makes sense; now HaikuDepot would have no reason at all to think it is the same as "master"; with just the symlink, the remote repo.info does claim to be the same as the "master" repository.

I guess HaikuDepot ignores the remote repo.info and just uses the local one? It would be nice to have a correct "identifier" here, but if we have to do what we did during beta1, I guess we will...

[apl-haiku]

Hi Augustine; If I pull down the R1B2 remote repo.info file I still seem to see the old repository identifier with "master" in the identifier? I thought you were going to swap the "master" for "r1beta2"?

[waddlesplash]

I did, and it didn't help, so I restored it back. The repository *is* that of master, so why does it make sense to identify it otherwise here? Won't that just confuse HDS more?

[apl-haiku]

If you switch the identifier over to the "r1beta2" in the remote repo.info file, HDS won't pick up on this until you execute the following API call to pull the repo.info data into HDS...

curl "https://depot.haiku-os.org/__repository/haikuportsr1b2/import

You can check this is configured into HDS properly by visiting ​here. It will take a few minutes for it to synchronize as it runs in a background thread after it is triggered. Keep checking until it has synchronized.

After this change is picked-up by HDS then the HaikuDepot client will check to look for an update when it starts and will pull down new repository data from HDS. Once this is done then it should marry-up the url (identifier) in HDS with the url identifier locally and it should work.

---

Won't that just confuse HDS more?

I just tried to draw all this into a diagram, but it is a bit hard to express. Let me try in words...

HDS has a set of "Repository Sources". Each time HDS syncs with the remote repo (1x daily and when prompted) it will pull down the repo.info and will update the url (identifier) in its local database record for the "Repository Source" -- if it has changed.

At start time, HaikuDepot will query HDS to see if it has newer data about the repositories. If it does have newer data, it will download it. This data includes the url identifier that was originally obtained from the sync with the remote repository's repo.info file above.

When HaikuDepot starts it loads the HPKR data from local disk and knows the repository that each package was associated with and hence knows the url (identifier) for each package.

HaikuDepot has a copy of HDS's data for packages which is updates as necessary. This data identifies the HDS repository involved. This means that HaikuDepot knows for the HDS data each package and knows the url (identifier) for each package.

It can now marry-up the repository cache package data and the HDS package data via the url (identifier) being equal.

---

So you need to ensure the local url}} (identifier) on the repository config is right and the remote repository {{{repo.info's url (identifier) is right, make sure HDS is synchronized with the remote repository and then restart HaikuDepot and it should marry-up the local package data with the HDS package data.

If this doesn't work then it be necessary to find where these two url (identifier) values do not match.

[waddlesplash]

But won't this separate the repository Featured data from master? Or is this treated as part of the "HaikuPorts" umbrella, so all packages with the same name are treated as the same across architectures, etc?

[apl-haiku]

Any meaningful data will be shared. Obviously versions of packages that appear in one repository are separated from the other repository, but otherwise icons, screenshots, localizations etc... will be shared.

[waddlesplash]

How does it know to share it? Are HaikuPorts and HaikuPorts R1B2 marked as being "the same" somehow? Or will any two packages of identical name from different repositories wind up being "equal"?

[waddlesplash]

I've configured the repositories and triggered the import. Not sure what to look for now though. HDS says "Primary/Active"...

[waddlesplash]

On the main page, the "R1B1" repository displays lower numbers, and clearly has no Featured information at all. Not sure what is to be done here...

[apl-haiku]

Or will any two packages of identical name from different repositories wind up being "equal"?

Yes if they have the same name then they share much of the data. See ​here for a data model which may help. Look for PkgSupplement and this material is shared off the packages -- effectively by their name.

[apl-haiku]

@waddlesplash you are looking for the url (identifier) changing. See highlighted area here...

If you re-import from the remote using the API...

curl "https://depot.haiku-os.org/__repository/haikuportsr1b2/import

...then it should update from the remote repo.info file. If it does not change can you look in the logs to see if there is a problem. I have no access to the deployment environment.

[waddlesplash]

Yes if they have the same name then they share much of the data.

That is really not good. Even within HaikuPorts, if we were to actually create two branches with different sets of packages, then e.g. a description/translation in one must be able to be different than in another, in case a package changes substantially between the two. Not to mention that other repositories must be able to provide alternative versions of packages with different metadata, too...

[waddlesplash]

Yes, I ran that "curl" command last night, and it said that it had been scheduled. As you can see, ​https://eu.hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current/repo.info indeed has an "identifier" of "...r1beta2..." now, but this does not appear to have helped.

[apl-haiku]

That is really not good. Even within HaikuPorts...

As far as I recall, this was all discussed many years ago. It has been working like this for 6-7 years. Data that is extracted from the HPKR is stored on a repository + package version level so is unique to the repository and version of the package. Data that is manually curated is stored at the package level. Changing this arrangement would take a considerable amount of time.

Yes, I ran that "curl" command last night, and it said that it had been scheduled. ...

1. Run the "curl" command
2. Observe the HDS logs coming from the Docker container
3. Copy paste the logs here and/or analyse the HDS logs to see what the problem is

[nielx]

Import Log

[nielx]

Exception:

2020-06-02T05:13:07.063641548Z 2020-06-02 05:13:07,062 / [job-run-3...] ERROR o.h.h.r.j.RepositoryHpkrIngressJobRunner - a problem has arisen processing a repository file for repository source [haikuportsr1b2_x86_gcc2]
2020-06-02T05:13:07.063684613Z org.apache.cayenne.access.OptimisticLockException: [v.4.0.2 Oct 09 2019 13:47:33] Optimistic Lock Failure, SQL: [UPDATE haikudepot.repository_source SET last_import_timestamp = ? WHERE id = ? AND active = ? AND code = ? AND forced_internal_base_url = ? AND url = ?], WHERE clause bindings: [code=haikuportsr1b2_x86_gcc2, active=true, forced_internal_base_url=http://www:80/haikuports/r1beta2/x86_gcc2/current, id=9184, url=https://hpkg.haiku-os.org/haikuports/master/x86_gcc2/current]
2020-06-02T05:13:07.063696242Z  at org.apache.cayenne.access.jdbc.BatchAction.runAsIndividualQueries(BatchAction.java:171)
2020-06-02T05:13:07.063702598Z  at org.apache.cayenne.access.jdbc.BatchAction.performAction(BatchAction.java:87)
2020-06-02T05:13:07.063708236Z  at org.apache.cayenne.access.DataNodeQueryAction.runQuery(DataNodeQueryAction.java:97)
2020-06-02T05:13:07.063713913Z  at org.apache.cayenne.access.DataNode.performQueries(DataNode.java:293)
2020-06-02T05:13:07.063719417Z  at org.apache.cayenne.access.DataDomainFlushAction.runQueries(DataDomainFlushAction.java:233)
2020-06-02T05:13:07.063724671Z  at org.apache.cayenne.access.DataDomainFlushAction.flush(DataDomainFlushAction.java:153)
2020-06-02T05:13:07.063729845Z  at org.apache.cayenne.access.DataDomain.onSyncFlush(DataDomain.java:633)
2020-06-02T05:13:07.063734972Z  at org.apache.cayenne.access.DataDomain.onSyncNoFilters(DataDomain.java:603)
2020-06-02T05:13:07.063740391Z  at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
2020-06-02T05:13:07.063745575Z  at org.apache.cayenne.tx.TransactionFilter$1.perform(TransactionFilter.java:73)
2020-06-02T05:13:07.063750593Z  at org.apache.cayenne.tx.TransactionFilter$1.perform(TransactionFilter.java:70)
2020-06-02T05:13:07.063755686Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:87)
2020-06-02T05:13:07.063760681Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:51)
2020-06-02T05:13:07.063766164Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:40)
2020-06-02T05:13:07.063771828Z  at org.apache.cayenne.tx.TransactionFilter.onSync(TransactionFilter.java:70)
2020-06-02T05:13:07.063780513Z  at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
2020-06-02T05:13:07.063786285Z  at org.haiku.haikudepotserver.support.cayenne.QueryCacheRemoveGroupDataChannelFilter.onSync(QueryCacheRemoveGroupDataChannelFilter.java:55)
2020-06-02T05:13:07.063803359Z  at org.apache.cayenne.access.DataDomain$DataDomainSyncFilterChain.onSync(DataDomain.java:764)
2020-06-02T05:13:07.063809796Z  at org.apache.cayenne.access.DataDomain.onSync(DataDomain.java:590)
2020-06-02T05:13:07.063814916Z  at org.apache.cayenne.access.DataContext.flushToParent(DataContext.java:748)
2020-06-02T05:13:07.063826807Z  at org.apache.cayenne.access.DataContext.commitChanges(DataContext.java:697)
2020-06-02T05:13:07.063843053Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.runForRepositorySource(RepositoryHpkrIngressJobRunner.java:119)
2020-06-02T05:13:07.063855277Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.lambda$run$1(RepositoryHpkrIngressJobRunner.java:97)
2020-06-02T05:13:07.063860751Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:87)
2020-06-02T05:13:07.063865852Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInLocalTransaction(DefaultTransactionManager.java:59)
2020-06-02T05:13:07.063870988Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:52)
2020-06-02T05:13:07.063876199Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:40)
2020-06-02T05:13:07.063881361Z  at org.apache.cayenne.configuration.server.ServerRuntime.performInTransaction(ServerRuntime.java:143)
2020-06-02T05:13:07.063887919Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.lambda$run$2(RepositoryHpkrIngressJobRunner.java:95)
2020-06-02T05:13:07.063893404Z  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
2020-06-02T05:13:07.063898413Z  at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
2020-06-02T05:13:07.063903367Z  at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
2020-06-02T05:13:07.063908753Z  at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
2020-06-02T05:13:07.063913845Z  at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
2020-06-02T05:13:07.063918972Z  at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2020-06-02T05:13:07.063926036Z  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2020-06-02T05:13:07.063931570Z  at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2020-06-02T05:13:07.063936956Z  at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
2020-06-02T05:13:07.063942013Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.run(RepositoryHpkrIngressJobRunner.java:94)
2020-06-02T05:13:07.063946793Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.run(RepositoryHpkrIngressJobRunner.java:54)
2020-06-02T05:13:07.063959070Z  at org.haiku.haikudepotserver.job.LocalJobServiceImpl.runSpecificationInCurrentThread(LocalJobServiceImpl.java:298)
2020-06-02T05:13:07.063965001Z  at org.haiku.haikudepotserver.job.LocalJobServiceImpl.lambda$createInternalJobBySubmittingToExecutor$8(LocalJobServiceImpl.java:256)
2020-06-02T05:13:07.063970485Z  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
2020-06-02T05:13:07.063975693Z  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2020-06-02T05:13:07.063996754Z  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2020-06-02T05:13:07.064003802Z  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2020-06-02T05:13:07.064009028Z  at java.base/java.lang.Thread.run(Thread.java:834)
2020-06-02T05:13:07.064781658Z 2020-06-02 05:13:07,064 / [job-run-3...] ERROR o.h.h.job.model.JobService - 3eef285f-bfbd-4a12-921b-fc436c5a2f19; failure to run the job
2020-06-02T05:13:07.064809532Z org.apache.cayenne.CayenneRuntimeException: [v.4.0.2 Oct 09 2019 13:47:33] java.lang.IllegalStateException: Transaction must have 'STATUS_ACTIVE' to be committed. Current status: STATUS_MARKED_ROLLEDBACK
2020-06-02T05:13:07.064816757Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInLocalTransaction(DefaultTransactionManager.java:68)
2020-06-02T05:13:07.064822775Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:52)
2020-06-02T05:13:07.064828231Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInTransaction(DefaultTransactionManager.java:40)
2020-06-02T05:13:07.064833991Z  at org.apache.cayenne.configuration.server.ServerRuntime.performInTransaction(ServerRuntime.java:143)
2020-06-02T05:13:07.064839450Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.lambda$run$2(RepositoryHpkrIngressJobRunner.java:95)
2020-06-02T05:13:07.064845639Z  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
2020-06-02T05:13:07.064851436Z  at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
2020-06-02T05:13:07.064857476Z  at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
2020-06-02T05:13:07.064863201Z  at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
2020-06-02T05:13:07.064868791Z  at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
2020-06-02T05:13:07.064877550Z  at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
2020-06-02T05:13:07.064883242Z  at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
2020-06-02T05:13:07.064889077Z  at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
2020-06-02T05:13:07.064894692Z  at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497)
2020-06-02T05:13:07.064909749Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.run(RepositoryHpkrIngressJobRunner.java:94)
2020-06-02T05:13:07.064916477Z  at org.haiku.haikudepotserver.repository.job.RepositoryHpkrIngressJobRunner.run(RepositoryHpkrIngressJobRunner.java:54)
2020-06-02T05:13:07.064923380Z  at org.haiku.haikudepotserver.job.LocalJobServiceImpl.runSpecificationInCurrentThread(LocalJobServiceImpl.java:298)
2020-06-02T05:13:07.064929416Z  at org.haiku.haikudepotserver.job.LocalJobServiceImpl.lambda$createInternalJobBySubmittingToExecutor$8(LocalJobServiceImpl.java:256)
2020-06-02T05:13:07.064934978Z  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
2020-06-02T05:13:07.064967339Z  at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2020-06-02T05:13:07.064972877Z  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2020-06-02T05:13:07.064978610Z  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2020-06-02T05:13:07.064984172Z  at java.base/java.lang.Thread.run(Thread.java:834)
2020-06-02T05:13:07.064989674Z Caused by: java.lang.IllegalStateException: Transaction must have 'STATUS_ACTIVE' to be committed. Current status: STATUS_MARKED_ROLLEDBACK
2020-06-02T05:13:07.064995655Z  at org.apache.cayenne.tx.BaseTransaction.commit(BaseTransaction.java:140)
2020-06-02T05:13:07.065001173Z  at org.apache.cayenne.tx.DefaultTransactionManager.performInLocalTransaction(DefaultTransactionManager.java:60)
2020-06-02T05:13:07.065006778Z  ... 22 common frames omitted

[apl-haiku]

Thank you for logs. It is very strange. I have spent quite a bit of time locally trying to reproduce this situation but am not able to unfortunately.

Could you please try to restart the docker container for HDS and then try the process again. It would also be interesting to know if there have been any other notable error log entries in the last week or two as that may shed some light on the issue.

Failing this, run the following SQL commands into the database to force the change in the records;

UPDATE haikudepot.repository_source
  SET url='https://hpkg.haiku-os.org/haikuports/r1beta2/x86_gcc2/current'
  WHERE code='haikuportsr1b2_x86_gcc2';
UPDATE haikudepot.repository_source
  SET url='https://hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current'
  WHERE code='haikuportsr1b2_x86_64';

Please also restart the HDS docker container after executing the SQL commands as the application server may have that data cached.

[nielx]

The restart seems to have done the trick. No error message and the repo list seems to be as expected.

[apl-haiku]

That's good in one way, but doesn't answer what is going on there. Are there any other errors that might indicate what the root cause is?

[nielx]

Don't think so:

2020-06-02 08:33:22,956 -/- [qtp405662939-18] INFO  o.h.h.job.model.JobService - js <9885..> (repositoryhpkringress); queued
2020-06-02 08:33:22,961 / [job-run-9...] INFO  o.h.h.job.model.JobService - js <9885..> (repositoryhpkringress); start
2020-06-02 08:33:23,043 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will import for repository source [RepositorySource[code=haikuportsr1b2_x86_gcc2]]
2020-06-02 08:33:23,438 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will copy data for repository info [RepositorySource[code=haikuportsr1b2_x86_gcc2]] (http://www:80/haikuports/r1beta2/x86_gcc2/current/repo.info) to temporary file
2020-06-02 08:33:23,861 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - updated the repo info url to [https://hpkg.haiku-os.org/haikuports/r1beta2/x86_gcc2/current] for repository source [haikuportsr1b2_x86_gcc2]
2020-06-02 08:33:23,969 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will copy repository hpkr [RepositorySource[code=haikuportsr1b2_x86_gcc2]] (http://www:80/haikuports/r1beta2/x86_gcc2/current/repo) to temporary file
2020-06-02 08:33:24,067 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did copy 1529274 bytes for repository hpkr [RepositorySource[code=haikuportsr1b2_x86_gcc2]] (http://www:80/haikuports/r1beta2/x86_gcc2/current/repo) to temporary file
2020-06-02 08:33:24,137 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will process data for repository hpkr haikuportsr1b2_x86_gcc2
2020-06-02 08:33:32,640 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.7.8..3] of package [yab_devel] did not exist; will create
2020-06-02 08:33:32,662 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [286832b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/yab_devel-1.7.8-3-x86_gcc2.hpkg]

2020-06-02 08:33:41,445 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.96.5..1] of package [acme_x86_source] did not exist; will create
2020-06-02 08:33:41,454 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [468854b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/acme_x86_source-0.96.5-1-source.hpkg]
2020-06-02 08:33:42,655 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.4...2] of package [yab_buildfactory] did not exist; will create
2020-06-02 08:33:42,666 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [27926b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/yab_buildfactory-2.4-2-x86_gcc2.hpkg]
2020-06-02 08:33:58,819 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.96.5..1] of package [acme_x86] did not exist; will create
2020-06-02 08:33:58,827 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [201608b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/acme_x86-0.96.5-1-x86_gcc2.hpkg]
2020-06-02 08:34:00,918 / [job-run-9...] WARN  o.h.h.pkg.PkgImportServiceImpl - imported a package version 1.18.2-1 of paramiko_python3 which is older or the same as the existing 2.6.0-1 -- will deactivate 1 pkg versions after the imported one and make the imported one as latest
2020-06-02 08:34:00,919 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - deactivated PkgVersion[versionCoordinates=2.6.0-1]
2020-06-02 08:34:04,831 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.16.2..4] of package [gst_plugins_bad_x86_devel] did not exist; will create
2020-06-02 08:34:04,838 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [94190b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/gst_plugins_bad_x86_devel-1.16.2-4-x86_gcc2.hpkg]
2020-06-02 08:34:18,280 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.3...1] of package [iff_catalog] did not exist; will create
2020-06-02 08:34:18,288 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [9898b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/iff_catalog-0.3-1-x86_gcc2.hpkg]
2020-06-02 08:34:22,418 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.7.8..3] of package [yab] did not exist; will create
2020-06-02 08:34:22,434 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [746881b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/yab-1.7.8-3-x86_gcc2.hpkg]
2020-06-02 08:34:23,341 / [job-run-9...] WARN  o.h.h.pkg.PkgImportServiceImpl - imported a package version 1.4.2-1 of slixmpp_python3 which is older or the same as the existing 1.5.2-1 -- will deactivate 1 pkg versions after the imported one and make the imported one as latest
2020-06-02 08:34:23,342 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - deactivated PkgVersion[versionCoordinates=1.5.2-1]
2020-06-02 08:34:24,937 / [job-run-9...] WARN  o.h.h.pkg.PkgImportServiceImpl - imported a package version 1.4.2-1 of slixmpp which is older or the same as the existing 1.5.2-1 -- will deactivate 1 pkg versions after the imported one and make the imported one as latest
2020-06-02 08:34:24,937 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - deactivated PkgVersion[versionCoordinates=1.5.2-1]
2020-06-02 08:34:25,416 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.16.2..4] of package [gst_plugins_bad_x86_source] did not exist; will create
2020-06-02 08:34:25,424 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [9801122b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/gst_plugins_bad_x86_source-1.16.2-4-source.hpkg]
2020-06-02 08:34:30,873 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.3...1] of package [iff_catalog_x86] did not exist; will create
2020-06-02 08:34:30,883 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [9963b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/iff_catalog_x86-0.3-1-x86_gcc2.hpkg]
2020-06-02 08:34:31,162 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.16.2..4] of package [gst_plugins_bad_x86] did not exist; will create
2020-06-02 08:34:31,170 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [7182217b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/gst_plugins_bad_x86-1.16.2-4-x86_gcc2.hpkg]
2020-06-02 08:34:39,754 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.3...1] of package [iff_catalog_source] did not exist; will create
2020-06-02 08:34:39,762 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [6967b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/iff_catalog_source-0.3-1-source.hpkg]
2020-06-02 08:34:44,003 / [job-run-9...] WARN  o.h.h.pkg.PkgImportServiceImpl - imported a package version 1.18.2-1 of paramiko_python which is older or the same as the existing 2.6.0-1 -- will deactivate 1 pkg versions after the imported one and make the imported one as latest
2020-06-02 08:34:44,003 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - deactivated PkgVersion[versionCoordinates=2.6.0-1]
2020-06-02 08:34:44,440 / [job-run-9...] WARN  o.h.h.pkg.PkgImportServiceImpl - imported a package version 1.18.2-1 of paramiko which is older or the same as the existing 2.6.0-1 -- will deactivate 1 pkg versions after the imported one and make the imported one as latest
2020-06-02 08:34:44,440 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - deactivated PkgVersion[versionCoordinates=2.6.0-1]
2020-06-02 08:34:46,761 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.7.8..3] of package [yab_source] did not exist; will create
2020-06-02 08:34:46,770 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [998836b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/yab_source-1.7.8-3-source.hpkg]
2020-06-02 08:34:48,694 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.3...1] of package [iff_catalog_x86_source] did not exist; will create
2020-06-02 08:34:48,701 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [6974b] for url [http://www:80/haikuports/r1beta2/x86_gcc2/current/packages/iff_catalog_x86_source-0.3-1-source.hpkg]
2020-06-02 08:34:49,770 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did process data for repository hpkr RepositorySource[code=haikuportsr1b2_x86_gcc2] in 85633ms
2020-06-02 08:34:49,781 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will import for repository source [RepositorySource[code=haikuportsr1b2_x86_64]]
2020-06-02 08:34:49,783 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will copy data for repository info [RepositorySource[code=haikuportsr1b2_x86_64]] (http://www:80/haikuports/r1beta2/x86_64/current/repo.info) to temporary file
2020-06-02 08:34:49,846 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - updated the repo info url to [https://hpkg.haiku-os.org/haikuports/r1beta2/x86_64/current] for repository source [haikuportsr1b2_x86_64]
2020-06-02 08:34:49,857 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will copy repository hpkr [RepositorySource[code=haikuportsr1b2_x86_64]] (http://www:80/haikuports/r1beta2/x86_64/current/repo) to temporary file
2020-06-02 08:34:49,953 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did copy 1303350 bytes for repository hpkr [RepositorySource[code=haikuportsr1b2_x86_64]] (http://www:80/haikuports/r1beta2/x86_64/current/repo) to temporary file
2020-06-02 08:34:49,954 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - will process data for repository hpkr haikuportsr1b2_x86_64
2020-06-02 08:34:53,157 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.2.11_20200514..1] of package [connectagram_source] did not exist; will create
2020-06-02 08:34:53,165 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [1693369b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/connectagram_source-1.2.11_20200514-1-source.hpkg]
2020-06-02 08:34:53,452 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [3.6.0..4] of package [embree_debuginfo] did not exist; will create
2020-06-02 08:34:53,465 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [550821b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/embree_debuginfo-3.6.0-4-x86_64.hpkg]
2020-06-02 08:34:55,069 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.2.11_20200514..1] of package [connectagram] did not exist; will create
2020-06-02 08:34:55,076 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [1404831b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/connectagram-1.2.11_20200514-1-x86_64.hpkg]
2020-06-02 08:34:55,087 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.2.5..2] of package [libopenshot_python3_debuginfo] did not exist; will create
2020-06-02 08:34:55,093 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [3900864b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libopenshot_python3_debuginfo-0.2.5-2-x86_64.hpkg]
2020-06-02 08:34:57,952 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.3.0..2] of package [gd] did not exist; will create
2020-06-02 08:34:57,959 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [482690b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/gd-2.3.0-2-x86_64.hpkg]
2020-06-02 08:35:03,563 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [7.1...3] of package [testdisk_qt] did not exist; will create
2020-06-02 08:35:03,571 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [1035870b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/testdisk_qt-7.1-3-x86_64.hpkg]
2020-06-02 08:35:06,550 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.7.6..1] of package [dmtx_utils_source] did not exist; will create
2020-06-02 08:35:06,557 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [57976b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/dmtx_utils_source-0.7.6-1-source.hpkg]
2020-06-02 08:35:07,462 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [7.1...3] of package [testdisk] did not exist; will create
2020-06-02 08:35:07,468 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [2495464b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/testdisk-7.1-3-x86_64.hpkg]
2020-06-02 08:35:08,035 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [5.1.0..4] of package [octave] did not exist; will create
2020-06-02 08:35:08,042 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [19044185b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/octave-5.1.0-4-x86_64.hpkg]
2020-06-02 08:35:08,680 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.7.8..3] of package [yab] did not exist; will create
2020-06-02 08:35:08,691 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [732067b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/yab-1.7.8-3-x86_64.hpkg]
2020-06-02 08:35:09,392 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.16.2..4] of package [gst_plugins_bad_source] did not exist; will create
2020-06-02 08:35:09,407 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [9800909b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/gst_plugins_bad_source-1.16.2-4-source.hpkg]
2020-06-02 08:35:09,944 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.8.0..5] of package [rdesktop] did not exist; will create
2020-06-02 08:35:09,961 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [112508b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/rdesktop-1.8.0-5-x86_64.hpkg]
2020-06-02 08:35:10,172 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.96.5..1] of package [acme_source] did not exist; will create
2020-06-02 08:35:10,185 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [468843b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/acme_source-0.96.5-1-source.hpkg]
2020-06-02 08:35:10,493 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.3.0..2] of package [gd_source] did not exist; will create
2020-06-02 08:35:10,500 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [2692772b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/gd_source-2.3.0-2-source.hpkg]
2020-06-02 08:35:10,949 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.2.5..2] of package [libopenshot_debuginfo] did not exist; will create
2020-06-02 08:35:10,955 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [16006102b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libopenshot_debuginfo-0.2.5-2-x86_64.hpkg]
2020-06-02 08:35:11,599 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.5.1..3] of package [imlib2] did not exist; will create
2020-06-02 08:35:11,606 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [507473b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/imlib2-1.5.1-3-x86_64.hpkg]
2020-06-02 08:35:12,921 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.26...7] of package [flickcurl_debuginfo] did not exist; will create
2020-06-02 08:35:12,931 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [605737b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/flickcurl_debuginfo-1.26-7-x86_64.hpkg]
2020-06-02 08:35:13,617 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.2.5..2] of package [libopenshot_python3] did not exist; will create
2020-06-02 08:35:13,633 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [521024b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libopenshot_python3-0.2.5-2-x86_64.hpkg]
2020-06-02 08:35:14,543 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.26...7] of package [flickcurl_devel] did not exist; will create
2020-06-02 08:35:14,548 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [1601285b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/flickcurl_devel-1.26-7-x86_64.hpkg]
2020-06-02 08:35:14,727 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.2.5..2] of package [libopenshot] did not exist; will create
2020-06-02 08:35:14,736 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [460088b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libopenshot-0.2.5-2-x86_64.hpkg]
2020-06-02 08:35:14,760 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.26...7] of package [flickcurl] did not exist; will create
2020-06-02 08:35:14,765 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [815796b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/flickcurl-1.26-7-x86_64.hpkg]
2020-06-02 08:35:14,805 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [3.6.0..4] of package [embree_source] did not exist; will create
2020-06-02 08:35:14,813 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [5598231b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/embree_source-3.6.0-4-source.hpkg]
2020-06-02 08:35:17,148 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.5.23..3] of package [libgphoto2_devel] did not exist; will create
2020-06-02 08:35:17,155 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [25910b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libgphoto2_devel-2.5.23-3-x86_64.hpkg]
2020-06-02 08:35:17,999 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.0.5_1..1] of package [streamradio] did not exist; will create
2020-06-02 08:35:18,004 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [118046b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/streamradio-0.0.5_1-1-x86_64.hpkg]
2020-06-02 08:35:18,427 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [5.1.0..4] of package [octave_devel] did not exist; will create
2020-06-02 08:35:18,438 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [546922b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/octave_devel-5.1.0-4-x86_64.hpkg]
2020-06-02 08:35:18,506 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.5.23..3] of package [libgphoto2] did not exist; will create
2020-06-02 08:35:18,519 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [3206766b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libgphoto2-2.5.23-3-x86_64.hpkg]
2020-06-02 08:35:18,609 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.2.5..2] of package [libopenshot_devel] did not exist; will create
2020-06-02 08:35:18,616 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [65221b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libopenshot_devel-0.2.5-2-x86_64.hpkg]
2020-06-02 08:35:19,563 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [3.6.0..4] of package [embree] did not exist; will create
2020-06-02 08:35:19,569 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [10704890b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/embree-3.6.0-4-x86_64.hpkg]
2020-06-02 08:35:20,119 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.16.2..4] of package [gst_plugins_bad_devel] did not exist; will create
2020-06-02 08:35:20,125 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [94142b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/gst_plugins_bad_devel-1.16.2-4-x86_64.hpkg]
2020-06-02 08:35:21,920 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.5.1..3] of package [imlib2_devel] did not exist; will create
2020-06-02 08:35:21,926 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [513143b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/imlib2_devel-1.5.1-3-x86_64.hpkg]
2020-06-02 08:35:22,950 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [7.1...3] of package [testdisk_source] did not exist; will create
2020-06-02 08:35:22,962 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [923865b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/testdisk_source-7.1-3-source.hpkg]
2020-06-02 08:35:24,059 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.0_git...6] of package [haikutwitter_source] did not exist; will create
2020-06-02 08:35:24,065 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [87281b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/haikutwitter_source-1.0_git-6-source.hpkg]
2020-06-02 08:35:26,660 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [8.2.0869..1] of package [vim] did not exist; will create
2020-06-02 08:35:26,665 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [13479947b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/vim-8.2.0869-1-x86_64.hpkg]
2020-06-02 08:35:29,442 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.0_git...6] of package [haikutwitter] did not exist; will create
2020-06-02 08:35:29,447 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [198209b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/haikutwitter-1.0_git-6-x86_64.hpkg]
2020-06-02 08:35:29,910 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [3.6.0..4] of package [embree_devel] did not exist; will create
2020-06-02 08:35:29,915 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [16856b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/embree_devel-3.6.0-4-x86_64.hpkg]
2020-06-02 08:35:30,949 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.5.1..3] of package [imlib2_source] did not exist; will create
2020-06-02 08:35:30,955 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [1058586b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/imlib2_source-1.5.1-3-source.hpkg]
2020-06-02 08:35:31,955 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.5.1..3] of package [imlib2_debuginfo] did not exist; will create
2020-06-02 08:35:31,963 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [414362b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/imlib2_debuginfo-1.5.1-3-x86_64.hpkg]
2020-06-02 08:35:34,456 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.7.8..3] of package [yab_devel] did not exist; will create
2020-06-02 08:35:34,462 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [287399b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/yab_devel-1.7.8-3-x86_64.hpkg]
2020-06-02 08:35:34,840 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.5.23..3] of package [libgphoto2_debuginfo] did not exist; will create
2020-06-02 08:35:34,845 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [183604b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libgphoto2_debuginfo-2.5.23-3-x86_64.hpkg]
2020-06-02 08:35:35,845 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.96.5..1] of package [acme] did not exist; will create
2020-06-02 08:35:35,850 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [211809b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/acme-0.96.5-1-x86_64.hpkg]
2020-06-02 08:35:35,942 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.3.0..2] of package [gd_devel] did not exist; will create
2020-06-02 08:35:35,947 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [608729b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/gd_devel-2.3.0-2-x86_64.hpkg]
2020-06-02 08:35:37,302 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [1.16.2..4] of package [gst_plugins_bad] did not exist; will create
2020-06-02 08:35:37,308 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [7793434b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/gst_plugins_bad-1.16.2-4-x86_64.hpkg]
2020-06-02 08:35:38,911 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.2.5..2] of package [libopenshot_source] did not exist; will create
2020-06-02 08:35:38,919 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [10343729b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libopenshot_source-0.2.5-2-source.hpkg]
2020-06-02 08:35:40,259 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [0.7.6..1] of package [dmtx_utils] did not exist; will create
2020-06-02 08:35:40,264 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [28618b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/dmtx_utils-0.7.6-1-x86_64.hpkg]
2020-06-02 08:35:41,557 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [8.2.0869..1] of package [vim_source] did not exist; will create
2020-06-02 08:35:41,562 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [15453826b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/vim_source-8.2.0869-1-source.hpkg]
2020-06-02 08:35:41,735 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [5.1.0..4] of package [octave_source] did not exist; will create
2020-06-02 08:35:41,744 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [33196560b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/octave_source-5.1.0-4-source.hpkg]
2020-06-02 08:35:42,457 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.4...2] of package [yab_buildfactory] did not exist; will create
2020-06-02 08:35:42,535 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [27911b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/yab_buildfactory-2.4-2-x86_64.hpkg]
2020-06-02 08:35:45,721 / [job-run-9...] INFO  o.h.h.pkg.PkgImportServiceImpl - the version [2.5.23..3] of package [libgphoto2_source] did not exist; will create
2020-06-02 08:35:45,727 / [job-run-9...] INFO  o.h.h.support.URLHelperService - did obtain length [9084534b] for url [http://www:80/haikuports/r1beta2/x86_64/current/packages/libgphoto2_source-2.5.23-3-source.hpkg]
2020-06-02 08:35:46,327 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did remove all versions of package qca2_devel from repository source RepositorySource[code=haikuportsr1b2_x86_64] because this package is no longer in the repository
2020-06-02 08:35:46,335 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did remove all versions of package qca2 from repository source RepositorySource[code=haikuportsr1b2_x86_64] because this package is no longer in the repository
2020-06-02 08:35:46,340 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did remove all versions of package qca2_source from repository source RepositorySource[code=haikuportsr1b2_x86_64] because this package is no longer in the repository
2020-06-02 08:35:46,343 / [job-run-9...] INFO  o.h.h.r.j.RepositoryHpkrIngressJobRunner - did process data for repository hpkr RepositorySource[code=haikuportsr1b2_x86_64] in 56389ms
2020-06-02 08:35:46,359 / [job-run-9...] INFO  o.h.h.job.model.JobService - js <9885..> (repositoryhpkringress); finish
2020-06-02 08:36:39,250 -/chr [qtp405662939-14] INFO  o.h.haikudepotserver.api1.PkgApiImpl - search for pkgs found 15 results
2020-06-02 08:36:39,649 -/chr [qtp405662939-55] INFO  o.h.h.s.AbstractExternalToolService - did create the temporary directory; /tmp/1591086999649-0
2020-06-02 08:36:39,944 -/chr [qtp405662939-56] INFO  o.h.h.s.AbstractExternalToolService - did create the temporary directory; /tmp/1591086999944-0
2020-06-02 08:37:51,193 -/- [qtp405662939-61] INFO  o.h.h.support.web.FallbackController - did redirect to a package for; qbittorrent_x86
2020-06-02 08:39:40,867 -/hd [qtp405662939-59] INFO  o.h.h.job.model.JobService - js <ceac..> (pkgiconexportarchive); queued
2020-06-02 08:39:40,867 -/hd [qtp405662939-59] INFO  o.h.h.job.model.JobService - js <ceac..> (pkgiconexportarchive); start
2020-06-02 08:39:40,892 -/hd [qtp405662939-12] INFO  o.h.h.job.model.JobService - js <03cf..> (referencedumpexport); queued
2020-06-02 08:39:40,893 -/hd [qtp405662939-12] INFO  o.h.h.job.model.JobService - js <03cf..> (referencedumpexport); start
2020-06-02 08:39:41,142 -/hd [qtp405662939-12] INFO  o.h.h.job.model.JobService - js <03cf..> (referencedumpexport); finish
2020-06-02 08:39:41,143 -/hd [qtp405662939-15] INFO  o.h.h.job.model.JobService - js <e43e..> (repositorydumpexport); queued
2020-06-02 08:39:41,143 -/hd [qtp405662939-15] INFO  o.h.h.job.model.JobService - js <e43e..> (repositorydumpexport); start
2020-06-02 08:39:41,249 -/hd [qtp405662939-15] INFO  o.h.h.job.model.JobService - js <e43e..> (repositorydumpexport); finish
2020-06-02 08:39:41,477 -/hd [qtp405662939-54] INFO  o.h.h.job.controller.JobController - did start async stream job data; 081c4914-9e02-4615-ab48-184aa106f299
2020-06-02 08:39:41,479 / [qtp405662939-54] INFO  o.h.h.job.controller.JobController - did complete async stream job data; 081c4914-9e02-4615-ab48-184aa106f299
2020-06-02 08:39:41,485 -/hd [qtp405662939-11] INFO  o.h.h.job.controller.JobController - did start async stream job data; e538782f-7eb7-4b1e-99f0-36f20dab1be6
2020-06-02 08:39:41,485 / [qtp405662939-11] INFO  o.h.h.job.controller.JobController - did complete async stream job data; e538782f-7eb7-4b1e-99f0-36f20dab1be6
2020-06-02 08:39:42,035 -/hd [qtp405662939-62] INFO  o.h.h.job.model.JobService - js <c562..> (pkgdumpexport); queued
2020-06-02 08:39:42,036 -/hd [qtp405662939-62] INFO  o.h.h.job.model.JobService - js <c562..> (pkgdumpexport); start
2020-06-02 08:39:42,116 -/hd [qtp405662939-62] INFO  o.h.h.pkg.job.PkgDumpExportJobRunner - will dump pkg versions for 5620 pkgs
2020-06-02 08:39:52,866 -/hd [qtp405662939-59] INFO  o.h.h.p.j.AbstractPkgResourceExportArchiveJobRunner - did produce report for 2775 entries in 11999ms
2020-06-02 08:39:52,867 -/hd [qtp405662939-59] INFO  o.h.h.job.model.JobService - js <ceac..> (pkgiconexportarchive); finish
2020-06-02 08:39:53,244 -/hd [qtp405662939-18] INFO  o.h.h.job.controller.JobController - did start async stream job data; fb8159fa-a9e3-41bf-8925-55b021ce612f
2020-06-02 08:39:53,488 / [qtp405662939-12] INFO  o.h.h.job.controller.JobController - did complete async stream job data; fb8159fa-a9e3-41bf-8925-55b021ce612f
2020-06-02 08:39:53,696 -/hd [qtp405662939-62] INFO  o.h.h.job.model.JobService - js <c562..> (pkgdumpexport); finish
2020-06-02 08:39:53,800 -/hd [qtp405662939-13] INFO  o.h.h.job.controller.JobController - did start async stream job data; d9d8bc18-9c33-44e5-9527-293510583c07
2020-06-02 08:39:53,832 / [qtp405662939-59] INFO  o.h.h.job.controller.JobController - did complete async stream job data; d9d8bc18-9c33-44e5-9527-293510583c07
2020-06-02 08:41:40,136 -/hd [qtp405662939-11] INFO  o.h.h.job.controller.JobController - did start async stream job data; fb8159fa-a9e3-41bf-8925-55b021ce612f
2020-06-02 08:41:40,380 -/hd [qtp405662939-56] INFO  o.h.h.job.controller.JobController - did start async stream job data; e538782f-7eb7-4b1e-99f0-36f20dab1be6
2020-06-02 08:41:40,381 / [qtp405662939-56] INFO  o.h.h.job.controller.JobController - did complete async stream job data; e538782f-7eb7-4b1e-99f0-36f20dab1be6
2020-06-02 08:41:40,530 / [qtp405662939-57] INFO  o.h.h.job.controller.JobController - did complete async stream job data; fb8159fa-a9e3-41bf-8925-55b021ce612f
2020-06-02 08:41:41,739 -/hd [qtp405662939-58] INFO  o.h.h.job.model.JobService - js <701e..> (pkgdumpexport); queued
2020-06-02 08:41:41,739 -/hd [qtp405662939-58] INFO  o.h.h.job.model.JobService - js <701e..> (pkgdumpexport); start
2020-06-02 08:41:41,852 -/hd [qtp405662939-58] INFO  o.h.h.pkg.job.PkgDumpExportJobRunner - will dump pkg versions for 5861 pkgs
2020-06-02 08:41:49,959 -/hd [qtp405662939-58] INFO  o.h.h.job.model.JobService - js <701e..> (pkgdumpexport); finish
2020-06-02 08:41:50,347 -/hd [qtp405662939-54] INFO  o.h.h.job.controller.JobController - did start async stream job data; e794504c-3fa5-42cd-8df9-67e56dda6df1
2020-06-02 08:41:50,441 / [qtp405662939-62] INFO  o.h.h.job.controller.JobController - did complete async stream job data; e794504c-3fa5-42cd-8df9-67e56dda6df1
2020-06-02 08:42:23,006 / [pool-3-thread-1] INFO  o.h.h.m.MaintenanceServiceImpl - did trigger hourly maintenance

Also this issue is about the 'Featured' tab not showing anything, and a recent beta2 build still exhibits that behaviour.

[apl-haiku]

Also this issue is about the 'Featured' tab not showing anything, and a recent beta2 build still exhibits that behaviour.

Making a couple of assumptions, execute the following commands and paste the result here;

cat "/system/settings/package-repositories/HaikuPorts"

cat "/boot/home/config/cache/HaikuDepot/repository-all_en.json.gz | gzip -d

The url (identifier) needs to match for it to work.

[nielx]

That might be an issue here, the url and identifier use ​https://hpkg.haiku-os.org... instead of ​https://eu.hpkg.haiku-os.org

[nielx]

I'm updating the build, this might have to do with my build being not too recent. I infer from the build script that the repo file might be fetched remotely during the build, so this could be the issue there.

[waddlesplash]

The identifier is supposed to be "hpkg." without the "eu." on the front; the same is true of the "master" repository on the nightlies.

[nielx]

I updated the database using the SQL in comment:31 and restarted the container (i.e. killed it).

Unfortunately it still does not show any featured packages with a recent build of r1beta2 (hrev54154_101)

[apl-haiku]

Unfortunately it still does not show any featured packages with...

I compiled and installed an R1B2 build, checked the url (identifiers) match, configured 1 package to promoted in HDS and it seems to appear for me?

[waddlesplash]

Wait, so the icons, translations, etc. are carried between packages, but the promotion status isn't? The entire point here was that the repositories are aliases of one another and should have all the same featured packages.

If HDS is incapable of handling that, then I guess I will revert the changes on the server to have the r1b2 repository have a separate identifier, and change the r1beta2 branch to have the "master" identifier, which is incorrect, but will have to do for now as it did during beta1.

We are really going to have to discuss and re-evaluate HDS and HaikuDepot design and architecture going forward.

[apl-haiku]

Wait, so the icons, tran...

Take the time to view and understand the ERD diagram which will hopefully show you how the data structure works. You can view that ​here.

The following *manually* curated data is stored per package by name (or derivative of the name to handle related package name suffixes);

• Iconography
• Screenshots
• Changelog
• Categorization
• Localizations

The following are stored (per package + per repository);

• Prominance (controls the "featured")

Everything arriving in the HPKR is stored (Per package-version + per repository) including copyrights, licenses, summary, description etc...

I don't think sharing the prominence between the repositories for a given package makes any sense at all. If you start a "My Own Repository" with 10 packages and HaikuPorts has 1000 packages you will want to handle prominence for your own repository completely differently. Even master repository for HaikuPorts will get many new packages over the lifespan of R1B2 and the prominences will likely end up being quite different after some months have passed and different packages are arriving in the master.

If HDS is incapable of handling that, the... ...and re-evaluate HDS and HaikuDepot design and architecture

I think calling the entire "design and architecture" of this system into question is blowing this out of proportion. I would suggest we close this ticket since it appears to be working properly now that the url (identifier) values match. Then we should move this discussion to the HDS mailing list and work through any remaining issues to get this setup. For example, I could potentially write a SQL script to seed the prominence values from "master" --> "r1b2" in this unusual "repository forking" situation.

[waddlesplash]

Even master repository for HaikuPorts will get many new packages over the lifespan of R1B2 and the prominences will likely end up being quite different after some months have passed and different packages are arriving in the master.

Please reread what I was saying above: The "master" and "r1beta2" repositories are identical, they are aliases of one another. This was also the case for R1/beta1 throughout its entire lifespan, and I expect it will probably be true for R1/beta2 throughout its entire lifespan. So it is *not* the case that the packages will be different after "some months"; they are at present the same, and would only diverge in the case that we actually branched HaikuPorts, which we do not intend to do for various reasons.

Again, the case of "repository aliases" is what actually needs to go on here. Since HDS is incapable of handling that, I will restore the R1/beta1 logic of just having the "identifier" URL remain as "master", which is kind of a hack, but it works.

[waddlesplash]

I don't think sharing the prominence between the repositories for a given package makes any sense at all.

It does not make sense to me to share all other data, but not prominence. But I have stated before that sharing data based purely on a "name" key across repositories itself does not make sense; ideally, no data would be shared across repositories. But this is a debate for a later time...

I think calling the entire "design and architecture" of this system into question is blowing this out of proportion.

I am not saying that just because of this one item, but yes, this is a discussion for the mailing list, probably.

I would suggest we close this ticket since it appears to be working properly now that the url (identifier) values match. ... seed the prominence values from "master" --> "r1b2" in this unusual "repository forking" situation.

The repositories are not forked, the idea here is just to leave that option open should we take it; this way, the local URLs and identifiers are for what the new repository "would" be. Since HDS cannot handle that, I will just revert the changes on the server to say "master" again, and also revert the changes in the r1beta2 branch to instead have an identifier of "master". But this ticket cannot be closed until that is done, since it is still unresolved.

[waddlesplash]

Did that in ​6a622cd71f47. Changing this ticket's milestone as appropriate.

[apl-haiku]

It does not make sense to me to share all other data, but not prominence.

If "BeSly" repository has the kritta package and "Andrew" repository also has the kritta package then they may both want to promote different packages. I can't see it working if the promotions are across all repositories the same for the package?

The repositories are not forked...

OK I missed that point. I have opened an HDS ​ticket to implement that, but yes for now best to use master as you suggest and I will endeavor to that done for the next beta.

[nielx]

Replying to waddlesplash:

Even master repository for HaikuPorts will get many new packages over the lifespan of R1B2 and the prominences will likely end up being quite different after some months have passed and different packages are arriving in the master.

Please reread what I was saying above: The "master" and "r1beta2" repositories are identical, they are aliases of one another. This was also the case for R1/beta1 throughout its entire lifespan, and I expect it will probably be true for R1/beta2 throughout its entire lifespan. So it is *not* the case that the packages will be different after "some months"; they are at present the same, and would only diverge in the case that we actually branched HaikuPorts, which we do not intend to do for various reasons.

Again, the case of "repository aliases" is what actually needs to go on here. Since HDS is incapable of handling that, I will restore the R1/beta1 logic of just having the "identifier" URL remain as "master", which is kind of a hack, but it works.

I would say that the next step in thinking about this would be to understand how the HaikuPorts project/team is thinking that they want to organize releases and development in the future.

Right now, the situation is, as you say, that the r1beta1 and r1beta2 repositories are effectively aliases to the master repository. As I understand it (correct me if I'm wrong), the r1beta1 repository was never 'forked', meaning that there is no effective permanent package collection for that release.

At some point there will be a release that we will want to give some sort of permanent package repository, and there will also become a point where the we would want to create a new package repository that exists in parallel to another one. With a 99 or 100% overlap in the beginning, but with a different build host, and slowly diverging over time.

As a thought experiment, let's say that there is a Haiku 1.0 released, Haiku 1.1 released, a 1.2 in dev and a 2.0 in dev. It is not unthinkable that even the minor releases make changes in the Haiku or POSIX API, which would mean that it is likely that there could be a HaikuPorts 1.0, HaikuPorts 1.1 and HaikuPorts Master (for 2.0 in dev). Will the HaikuPorts 1.0 still receive updates? Probably yes, at least security updates. But what if 2.0 beta 1 comes out? What is HaikuPort's plan there? Will there be a permanent branch, or will we treat beta releases as throwaways once the final release comes out.

With the view and plans of our most important repository (HaikuPorts), it then becomes a lot easier to distill the requirements for HDS and HaikuDepot, also for the longer term. Right now I feel the requirements for HDS and HaikuDepot are not well-defined (or at least: not explicitly) and as such we get into these communication issues where het HDS design assumptions break the thing you as release manager expect to work in a certain way.

Two loose thoughts:

1. I would also tie this in with PulkoMandy's discussion about changing the repo identifier.
2. Right now we are in some sort of prerelease limbo. From a view point of persistency and saving your history, one could say it was wrong not to have a permanent package collection for r1beta1. From a practical perspective, I don't boot up r1alpha2 every now and then for nostalgia reasons, and likewise, I think that any R1 prereleases will be gathering dust once R1 is out.

[apl-haiku]

this would be to understand how the HaikuPorts project/team is thinking that they want to organize releases and development

Yes this makes good sense; let's get a plan together and see how it can work better for beta 3.