Testing sqlite

[mcguirepr89]

Howdy, gang --

I wanted to invite anyone that may be interested to download and test the new SQLite version of BirdNET-Pi.

This branch has the following changes:

• MariaDB has been replaced with SQLite
  • This means the entire database is just one file: /home/pi/BirdNET-Pi/scripts/birds.db -- very easy for backing up, migrating to new installations, and copying for import into data visualization programs.
• All of the views have been updated to be more interactive and in some places, informative

Overview

What is clickable:

• The number of detections for "Today" can be clicked to take the user to the "By_Date" view for the current date
• The number for "Species Detected Today" can be clicked to take the user to the "Species Stats" page
• The name for "Scientific Name" will take the user to the Wikipedia page for the binomial scientific name
• The name for "Common Name" will take the user to the "Species Stats" page while also submitting that species for a stats report
• The date and time for "Listen" can be clicked to load the extraction in the "Play Live Audio" frame in the top right corner

Database

What is clickable:

• The number of detections for "Today" can be clicked to take the user to the "By_Date" view for the current date
• The "Number of Unique Species" can be clicked to take the user to the "Species Stats" page
• The timestamp for each database entry can be clicked to load (play) the sound file in the top right "Live Audio Stream" frame
• The Scientific Name for each entry can be clicked to take the user to the Wikipedia page for the binomial scientific name
• The Common Name can be clicked to take the user to the All About Birds page for the species (in English)

Species Stats

• The Common Names can be clicked to load the species stats for the species
  

By Species

This page is aesthetic garbage, but functionally, it allows digging into the files By-Species using the database instead of symbolic links.

What is clickable:

• The timestamps load the audio as described for links above
• The scientific name loads the wikipedia for the species
• The common name submits the species to the "Species Stats" page and runs the report for that species

In terms of feedback from testing, I'm hoping to get reports on whether the system seems to be running more smoothly on lower-resourced hardware and what your thoughts are on the new user interface tweaks.

Thanks in advance to anyone with the time to test.
Please let me know if you have any trouble at all and I will be glad to help.
The installation one liner is as follows:

curl -s https://raw.githubusercontent.com/mcguirepr89/BirdNET-Pi/sqlite/newinstaller.sh | bash

My best regards as always,
Patrick

[Svardsten53]

Wow, a lot of news! I'll see if I can test it later today. The problem is that I will not have any detections when it's dark outside. Is there any way to restore the old database into the new one?

[oliverk71]

At least, the content of the MariaDB can be exported into a file. So, if sqlite allows import from file, it is generally possible.

[oliverk71]

I tried, but I did not manage to import the exported gz file. Most probably because I do not know the correct settings for export. Maybe @mcguirepr89 can give us a hint?

[DD4WH]

@mcguirepr89
Patrick, superb!
Installed the new version on a Raspberry Pi Zero 2W: first observations:

• no problems at all with installation on OS ARM64 lite, installation was quite quick (about 20-25minutes)
• seems to work very well even with standard settings
• new features are really awesome, especially to be able to click on a detection in the database view and play the specified recording!!! no more need to go through all the folders! That is my highlight in the new version!
• will test longer and report here (or do you want me to report specific feedback on the Zero 2W in the issue ?)

UPDATE - the Zero gets quite hot, 65 degrees

[Svardsten53]

I have also installed the SQLite version now, took about six minutes on the prepared SD card on my RPi 4B. It looks very nice! I have some findings:

1. After installation I got this error message. Reason could be there was nothing in the database. I went out in the dark and played some bird songs and the messages disappeared.

2. Text on the overview page is not centered as in your example. I use Firefox, but same result in Chrome.

3. Extractions on the left only gives a white page for "By Common Name" and "By Scientific Name", folders are missing in the file system.

4. VNC was not enabled in the installation. After enabling it in raspi-config I was able to reach RPi from VNC.

5. "Play Live Audio" results in "404 - The file you requested could not be found", but I have had this in my old installation for a couple of days, so have to find whats wrong.

You can reach the installation from the Internet and it will be up for a couple of hours if you want to take a look. Later I will reboot from the old SD card.

Nice work Patrick!!!

[mcguirepr89]

Hi, @Svardsten53! Thanks for giving it a go!
It looks as though for steps 1 and 2 you may be viewing http://birdnetpi.local/overview.php directly, instead of http://birdnetpi.local with overview.php in the "content" frame. Can you confirm that for me? If that is the case, then that is why the tables are not centered properly, as they take into account the other frames.

3. Extractions on the left only gives a white page for "By Common Name" and "By Scientific Name", folders are missing in the file system.

The SQLite system does not utilize the "By_Common" nor "By_Scientific" symlinked directory structure -- instead, PHP and database requests will mimic those views dynamically at play.php which will respond to the links in menu.html that POST form button submissions for either byspecies or bydate. The links in menu.html shouldn't point to those directories anymore, but I forgot to push my changes to menu.html so your links don't reflect that! Sorry! You should be able to just git pull to get the changes: UPDATE: I also forgot to symlink the new play.php to the ${EXTRACTIONS} directory:

git pull
ln -sf /home/pi/BirdNET-Pi/scripts/play.php /home/pi/BirdSongs/Extracted
ln -sf /home/pi/BirdNET-Pi/scripts/play.css /home/pi/BirdSongs/Extracted

and if git pull gives you issues:

git stash
git pull
git stash pop

VNC was not enabled in the installation. After enabling it in raspi-config I was able to reach RPi from VNC.

I guess it removed that from installation since I don't see a need for it for the average user -- I will, however, add enabling/disabling VNC in my future page for managing services. The current "Manage Services" page is also very unsophisticated and does not address all services.

"Play Live Audio" results in "404 - The file you requested could not be found", but I have had this in my old installation for a couple of days, so have to find whats wrong.

Not sure what could be going on there, but take a look in your "System" > "Info" page and at the very bottom is a services list you'll want to be certain shows all services running.

Thank you so much, Lars!! I appreciate the feedback and quick testing!~

[Svardsten53]

1 and 2: I have just run the site as it is, no url modifications and still have the problem with non centered tables on my desktop. When I use my laptop it looks much better.

3: The commands solved it. I have also run an "Update" and everything works fine.

"Play Live Audio": The info page says ffmpeg is off.

I will soon boot my old system and maybe continue with the SQLite version tomorrow night.

[mcguirepr89]

Thanks a lot, Lars! I saw you were able to update the site and it is looking great! It is possible to migrate the old database, but I will save that process for another post.

My very best regards to you and the cranes :)

[Svardsten53]

Thank you! :-)

Suddenly ffmpeg service is on and "Play Live Audio" works!

[DD4WH]

the photographs of the species are really funny sometimes:

:-)

[mcguirepr89]

hahaha yeah, I need a bird-image database to use, but I don't know of any that are available :/
The image fetching is really unsophisticated and just goes with whatever the first result is for the binomial scientific name for a wiki commons image search.

I would love, though, for it to be more sophisticated and/or pull from a real bird-image database.

[oliverk71]

What about flickr? The search can be filtered by license.

[nilspupils]

Installed and did't have any issues at all with quick testing!
Thank you for another great update!!

[mcguirepr89]

Awesome to hear! Thank you so much, Nils!

[mcguirepr89]

You all please do let me know where I should make changes to the linking and navigation, etc.

[DD4WH]

Patrick, thanks again for this new version!
Navigation and links are fine and very intuitive!

Here are some comments/observations (Lite install on RPi 4B):

• Is there a possibility to color those entries that are "clickable" to distinguish them from the others? (Maybe this is not so important because the cursor change indicates clickable links)
• the Live Stream is not reachable any more (without typing in the URL: birdnetpi.local/stream) once I clicked on "Listen" in the Overview where the most recent recording is shown (or if I click on the leftmost column in the database view ("Time"))
• Confidence score should always be rounded to one decimal digit and expressed in % (eg. 92.5%) --> on all pages, but also in the LOG
• Species Stats page should print the time frame for this statistic, eg. ("Birds detected from 10-02-2022 to 23-02-2022 are shown")

Keep up the good work!

[oliverk71]

Nice, I would like to test the new SQLite birdnet-pi. Can I switch from a 'main' installation to 'sqlite' or do I have to do a clean install?

[mcguirepr89]

Thank, Oliver! Please test a new installation and please have no expectations of migrating to main from the sqlite branch.
My best,
Patrick

[oliverk71]

I just installed and ....

[oliverk71]

Another, shorter log excerpt:

00:15:41 birdnet_analysis.sh[741383]: python3 analyze.py --i /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-00:15:26.wav --o /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-00:15:26.wav.csv --lat 51.0914 --lon 6.5716 --week 8 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7 --exclude_list /home/pi/BirdNET-Pi/exclude_species_list.txt --birdweather_id IN_USE

[mcguirepr89]

Thanks, Oliver -- I found the issue with your menu.html links, which is that the symbolic links for the the play.php and play.css weren't made properly. I fixed that part of the installation script, so thank you very much for your help! I don't yet know why your log is showing the BIRDWEATHER_ID when there is none, so I will keep looking into that.

A quick fix for your menu.html "By Date" and "By Species" buttons:

ln -sf /home/pi/BirdNET-Pi/scripts/play.php /home/pi/BirdSongs/Extracted
ln -sf /home/pi/BirdNET-Pi/scripts/play.css /home/pi/BirdSongs/Extracted

[mcguirepr89]

Hi, @oliverk71 -- I've looked through the scripts more and more and am not yet convinced that the system is malfunctioning in regards to the BirdWeather ID. I wanted to check with you to be 100% certain that the logs you are viewing are definitely for the new installation that does not use the BirdWeather ID and are not the logs available on port 8080 for your other installation. Mixing up those logs is the only explanation I can come up with as to why you would be seeing the BirdWeather ID in use in the log -- the only way the log mentioned using the BirdWeather ID is when there is indeed a BirdWeather ID variable that is not empty. It cannot be verbose about that variable otherwise.

Please double check and be absolutely certain that you are viewing the log for the new installation and not your other installation.

Thank you in advance for your help in clarifying this,

Patrick

[oliverk71]

Guess, you are right. The menu link points to birdnetpi.local:8080 which could explain that the logs are possibly mixed. I checked 192.168.1.7:8080 and I cannot see anything about birdweather ID, now. Sorry for confusion.

15:35:56 server.py[914542]: WRITING RESULTS TO /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-15:35:26.wav.csv ... DONE! WROTE 0 RESULTS.
15:35:56 server.py[914542]: [NEW CONNECTION] ('127.0.0.1', 40170) connected.
15:35:56 server.py[914542]: [ACTIVE CONNECTIONS] 1
15:35:56 server.py[914542]: READING AUDIO DATA... DONE! READ 5 CHUNKS.
15:35:56 server.py[914542]: FULL FILENAME: -/home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-15:35:41.wav-
15:35:56 server.py[914542]: Date-time: 2022-02-24 15:35:41
15:36:00 birdnet_analysis.sh[964490]: 0.0;3.0-('Pica pica_Elster', 0.0824193)
15:36:00 birdnet_analysis.sh[964490]: 3.0;6.0-('Sylvia atricapilla_Mönchsgrasmücke', 0.061978832)
15:36:00 birdnet_analysis.sh[964490]: 6.0;9.0-('Picoides tridactylus_Dreizehenspecht', 0.08306276)
15:36:00 birdnet_analysis.sh[964490]: 9.0;12.0-('Caprimulgus europaeus_Ziegenmelker', 0.053939555)
15:36:00 birdnet_analysis.sh[964490]: 12.0;15.0-('Bubo bubo_Uhu', 0.056806516)
15:36:00 systemd[1]: birdnet_analysis.service: Succeeded.
15:36:00 systemd[1]: birdnet_analysis.service: Consumed 4.192s CPU time.
15:36:02 systemd[1]: birdnet_analysis.service: Scheduled restart job, restart counter is at 3376.
15:36:02 systemd[1]: Stopped BirdNET Analysis.
15:36:02 systemd[1]: birdnet_analysis.service: Consumed 4.192s CPU time.
15:36:02 systemd[1]: Started BirdNET Analysis.
15:36:04 birdnet_analysis.sh[964583]: Starting run_birdnet() for February-2022/24-Thursday
15:36:04 birdnet_analysis.sh[964583]: get_files() for February-2022/24-Thursday
15:36:04 birdnet_analysis.sh[964583]: Files loaded
15:36:04 birdnet_analysis.sh[964583]: Starting move_analyzed() for February-2022/24-Thursday
15:36:04 birdnet_analysis.sh[964583]: Moving analyzed files to new directory
15:36:04 birdnet_analysis.sh[964583]: Starting run_analysis() for February-2022/24-Thursday
15:36:07 birdnet_analysis.sh[964583]: RECORDING_LENGTH set to 15
15:36:11 birdnet_analysis.sh[964583]: analyze.py --i /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-15:35:56.wav --o /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-15:35:56.wav.csv --lat 51.0837 --lon 6.6114 --week 8 --overlap 0.0 --sensitivity 1.25 --min_conf 0.7
15:36:11 server.py[914542]: ANALYZING AUDIO... DONE! Time 3.8 SECONDS
15:36:11 server.py[914542]: WRITING RESULTS TO /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-15:35:41.wav.csv ... DONE! WROTE 0 RESULTS.
15:36:11 server.py[914542]: [NEW CONNECTION] ('127.0.0.1', 40172) connected.
15:36:11 server.py[914542]: [ACTIVE CONNECTIONS] 1
15:36:11 server.py[914542]: READING AUDIO DATA... DONE! READ 5 CHUNKS.
15:36:11 server.py[914542]: FULL FILENAME: -/home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-15:35:56.wav-
15:36:11 server.py[914542]: Date-time: 2022-02-24 15:35:56
15:36:15 birdnet_analysis.sh[964749]: 0.0;3.0-('Bubo bubo_Uhu', 0.07013959)
15:36:15 birdnet_analysis.sh[964749]: 3.0;6.0-('Bubo bubo_Uhu', 0.055832293)
15:36:15 birdnet_analysis.sh[964749]: 6.0;9.0-('Bubo bubo_Uhu', 0.055284556)
15:36:15 birdnet_analysis.sh[964749]: 9.0;12.0-('Bubo bubo_Uhu', 0.05549699)
15:36:15 birdnet_analysis.sh[964749]: 12.0;15.0-('Coturnix coturnix_Wachtel', 0.05177428)
15:36:15 systemd[1]: birdnet_analysis.service: Succeeded.
15:36:15 systemd[1]: birdnet_analysis.service: Consumed 4.265s CPU time.

[mcguirepr89]

Glad to hear that there is not an issue with that 👍

[DD4WH]

After an update (about 30min ago) live stream is no longer working.
Info shows ffmpeg and icecast2 is red (status ==OFF)
Log shows normal behavior.
Also noticed that link for the log has not been properly updated after I changed the url in the settings yesterday (still points to birdnetpi.local:8080)

[DD4WH]

No, I never had to enter a password for an update, sounds strange

[DD4WH]

But, in a recent post, Patrick said that the update function in v0.12 is not yet working like it should. So better don't use it unless you have to.

[mcguirepr89]

One question. While updating, I was asked for the password for the pi user. Is this how it should be?

Hi, @oliverk71, as always, I will have to ask you exactly what command you issued prior to getting that prompt and any an all context that can help me figure out what is going on.
Thank you!

Update:
I think I may have figured out (guessed) what you have done and where the code needed to be tweaked.
Please try this again:

git stash
git pull
git stash pop
sudo ln -sf /home/pi/BirdNET-Pi/scripts/* /usr/local/bin
update_caddyfile.sh
source /etc/birdnet/birdnet.conf &&
ln -fs ~/BirdNET-Pi/scripts/play.css ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/play.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/spectrogram.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/overview.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/stats.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/viewdb.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/history.php ${EXTRACTED}
sudo systemctl daemon-reload
restart_services.sh

[oliverk71]

I used these commands to update:

git stash
git pull
git stash pop
sudo ln -sf /home/pi/BirdNET-Pi/scripts/* /usr/local/bin
update_caddyfile.sh
source /etc/birdnet/birdnet.conf &&
ln -fs ~/BirdNET-Pi/scripts/play.css ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/play.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/spectrogram.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/overview.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/stats.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/viewdb.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/history.php ${EXTRACTED}
sudo systemctl daemon-reload
restart_services.sh

The whole input/output:

git stash
git pull
git stash pop
sudo ln -sf /home/pi/BirdNET-Pi/scripts/* /usr/local/bin
update_caddyfile.sh
source /etc/birdnet/birdnet.conf &&
ln -fs ~/BirdNET-Pi/scripts/play.css ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/play.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/spectrogram.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/overview.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/stats.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/viewdb.php ${EXTRACTED}
ln -fs ~/BirdNET-Pi/scripts/history.php ${EXTRACTED}
sudo systemctl daemon-reload
restart_services.sh
Saved working directory and index state WIP on sqlite: 066b9c1 making sure play.php and play.css make their way into ${EXTRACTED}
hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
remote: Enumerating objects: 27, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 16 (delta 15), reused 15 (delta 14), pack-reused 0
Unpacking objects: 100% (16/16), 1.99 KiB | 39.00 KiB/s, done.
From https://github.com/mcguirepr89/BirdNET-Pi
   066b9c1..0190c0a  sqlite     -> origin/sqlite
 * [new branch]      newif      -> origin/newif
Updating 066b9c1..0190c0a
Fast-forward
 scripts/viewdb.php | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
On branch sqlite
Your branch is up to date with 'origin/sqlite'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   homepage/menu.html
	modified:   model/labels.txt

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	BirdDB.txt
	analyzing_now.txt
	birdnet/
	exclude_species_list.txt
	firstrun.ini
	include_species_list.txt
	lastrun.txt
	scripts/BirdDB.txt
	scripts/birds.db
	scripts/labels.txt
	thisrun.txt

no changes added to commit (use "git add" and/or "git commit -a")
Dropped refs/stash@{0} (4cda1dd7a10036ea54c9105194d706ddce797d39)
+ '[' -d /etc/caddy ']'
+ '[' -f /etc/caddy/Caddyfile ']'
+ cp /etc/caddy/Caddyfile /etc/caddy/Caddyfile.original
cp: cannot create regular file '/etc/caddy/Caddyfile.original': Permission denied
+ '[' -z ']'
+ cat
++ hostname
/usr/local/bin/update_caddyfile.sh: line 34: /etc/caddy/Caddyfile: Permission denied
+ '[' '!' -z ']'
+ '[' '!' -z ']'
+ '[' '!' -z ']'
+ sudo -upi git -C /home/pi/BirdNET-Pi checkout -f homepage/external.html homepage/footer.html homepage/images homepage/index.html homepage/menu_de.html homepage/menu.html homepage/mobile.html homepage/style.css homepage/top_de.html homepage/top.html
Updated 1 path from the index
+ sudo -upi git -C /home/pi/BirdNET-Pi checkout -f scripts/add_to_exclude.php scripts/add_to_include.php scripts/adminer-de.php scripts/adminer-fr.php scripts/adminer.php scripts/advanced.php scripts/analyze.py scripts/backup_inprogress.html scripts/BirdDB.txt scripts/birdnet_analysis.sh scripts/birdnet_log.sh scripts/birdnet-pi-config scripts/birdnet_recording.sh scripts/birds.db scripts/birdweather_request.php scripts/cleanup.sh scripts/clear_all_data.sh scripts/config.php scripts/createdb.sh scripts/daily_plot.py scripts/daily_plot.py.newer scripts/del_from_exclude.php scripts/del_from_include.php scripts/disk_check.sh scripts/disk_usage.sh scripts/dump_logs.sh scripts/email_template scripts/exclude_list.php scripts/extract_new_birdsounds.sh scripts/extra_info.sh scripts/filemanager scripts/first_boot.php scripts/get_citation.sh scripts/get_image.sh scripts/history.php scripts/include_list.php scripts/index.html scripts/install_birdnet.sh scripts/install_config.sh scripts/install_noip2.sh scripts/install_services.sh scripts/install_zram_service.sh scripts/labels.txt scripts/livestream.sh scripts/overview.php scripts/play.css scripts/play.php scripts/rebuild_db.sh scripts/restart_services.sh scripts/rewrite_config.sh scripts/server.py scripts/service_controls.php scripts/species_notifier.sh scripts/spectrogram.php scripts/spectrogram.sh scripts/stats.php scripts/stop_core_services.sh scripts/system_controls.php scripts/uninstall.sh scripts/update_birdnet.sh scripts/update_caddyfile.sh scripts/update_db_pwd_bullseye.sh scripts/update_inprogress.php scripts/update_settings_inprogress.php scripts/update_species.sh scripts/viewdb.php scripts/welcome_wizard.sh scripts/write_config.php scripts/write_config.sh
error: pathspec 'scripts/BirdDB.txt' did not match any file(s) known to git
error: pathspec 'scripts/birds.db' did not match any file(s) known to git
error: pathspec 'scripts/labels.txt' did not match any file(s) known to git
+ sudo -upi git -C /home/pi/BirdNET-Pi checkout -f scripts/filemanager/filemanager.php scripts/filemanager/LICENSE scripts/filemanager/SECURITY.md scripts/filemanager/translation.json
Updated 0 paths from the index
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/https:\/\/v2.wttr.in\//https:\/\/v2.wttr.in\/51.0837,6.6114/g' /home/pi/BirdNET-Pi/homepage/menu.html
+ '[' '!' -z ']'
++ sed 's/\/\//\\\/\\\//g'
+++ hostname
++ echo http://birdnetpi.local:8080
+ BIRDNETLOG_URL='http:\/\/birdnetpi.local:8080'
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8080/http:\/\/birdnetpi.local:8080/g' /home/pi/BirdNET-Pi/homepage/external.html /home/pi/BirdNET-Pi/homepage/footer.html /home/pi/BirdNET-Pi/homepage/index.html /home/pi/BirdNET-Pi/homepage/menu_de.html /home/pi/BirdNET-Pi/homepage/menu.html /home/pi/BirdNET-Pi/homepage/mobile.html /home/pi/BirdNET-Pi/homepage/top_de.html /home/pi/BirdNET-Pi/homepage/top.html
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8080/http:\/\/birdnetpi.local:8080/g' /home/pi/BirdNET-Pi/scripts/backup_inprogress.html /home/pi/BirdNET-Pi/scripts/index.html
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8080/http:\/\/birdnetpi.local:8080/g' /home/pi/BirdNET-Pi/scripts/backup_inprogress.html /home/pi/BirdNET-Pi/scripts/index.html
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8080/http:\/\/birdnetpi.local:8080/g' /home/pi/BirdNET-Pi/scripts/add_to_exclude.php /home/pi/BirdNET-Pi/scripts/add_to_include.php /home/pi/BirdNET-Pi/scripts/adminer-de.php /home/pi/BirdNET-Pi/scripts/adminer-fr.php /home/pi/BirdNET-Pi/scripts/adminer.php /home/pi/BirdNET-Pi/scripts/advanced.php /home/pi/BirdNET-Pi/scripts/birdweather_request.php /home/pi/BirdNET-Pi/scripts/config.php /home/pi/BirdNET-Pi/scripts/del_from_exclude.php /home/pi/BirdNET-Pi/scripts/del_from_include.php /home/pi/BirdNET-Pi/scripts/exclude_list.php /home/pi/BirdNET-Pi/scripts/first_boot.php /home/pi/BirdNET-Pi/scripts/history.php /home/pi/BirdNET-Pi/scripts/include_list.php /home/pi/BirdNET-Pi/scripts/overview.php /home/pi/BirdNET-Pi/scripts/play.php /home/pi/BirdNET-Pi/scripts/service_controls.php /home/pi/BirdNET-Pi/scripts/spectrogram.php /home/pi/BirdNET-Pi/scripts/stats.php /home/pi/BirdNET-Pi/scripts/system_controls.php /home/pi/BirdNET-Pi/scripts/update_inprogress.php /home/pi/BirdNET-Pi/scripts/update_settings_inprogress.php /home/pi/BirdNET-Pi/scripts/viewdb.php /home/pi/BirdNET-Pi/scripts/write_config.php
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8080/http:\/\/birdnetpi.local:8080/g' /home/pi/BirdNET-Pi/scripts/filemanager/filemanager.php
+ '[' '!' -z ']'
++ sed 's/\/\//\\\/\\\//g'
+++ hostname
++ echo http://birdnetpi.local:8888
+ WEBTERMINAL_URL='http:\/\/birdnetpi.local:8888'
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8888/http:\/\/birdnetpi.local:8888/g' /home/pi/BirdNET-Pi/homepage/external.html /home/pi/BirdNET-Pi/homepage/footer.html /home/pi/BirdNET-Pi/homepage/index.html /home/pi/BirdNET-Pi/homepage/menu_de.html /home/pi/BirdNET-Pi/homepage/menu.html /home/pi/BirdNET-Pi/homepage/mobile.html /home/pi/BirdNET-Pi/homepage/top_de.html /home/pi/BirdNET-Pi/homepage/top.html
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8888/http:\/\/birdnetpi.local:8888/g' /home/pi/BirdNET-Pi/scripts/backup_inprogress.html /home/pi/BirdNET-Pi/scripts/index.html
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8888/http:\/\/birdnetpi.local:8888/g' /home/pi/BirdNET-Pi/scripts/backup_inprogress.html /home/pi/BirdNET-Pi/scripts/index.html
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8888/http:\/\/birdnetpi.local:8888/g' /home/pi/BirdNET-Pi/scripts/add_to_exclude.php /home/pi/BirdNET-Pi/scripts/add_to_include.php /home/pi/BirdNET-Pi/scripts/adminer-de.php /home/pi/BirdNET-Pi/scripts/adminer-fr.php /home/pi/BirdNET-Pi/scripts/adminer.php /home/pi/BirdNET-Pi/scripts/advanced.php /home/pi/BirdNET-Pi/scripts/birdweather_request.php /home/pi/BirdNET-Pi/scripts/config.php /home/pi/BirdNET-Pi/scripts/del_from_exclude.php /home/pi/BirdNET-Pi/scripts/del_from_include.php /home/pi/BirdNET-Pi/scripts/exclude_list.php /home/pi/BirdNET-Pi/scripts/first_boot.php /home/pi/BirdNET-Pi/scripts/history.php /home/pi/BirdNET-Pi/scripts/include_list.php /home/pi/BirdNET-Pi/scripts/overview.php /home/pi/BirdNET-Pi/scripts/play.php /home/pi/BirdNET-Pi/scripts/service_controls.php /home/pi/BirdNET-Pi/scripts/spectrogram.php /home/pi/BirdNET-Pi/scripts/stats.php /home/pi/BirdNET-Pi/scripts/system_controls.php /home/pi/BirdNET-Pi/scripts/update_inprogress.php /home/pi/BirdNET-Pi/scripts/update_settings_inprogress.php /home/pi/BirdNET-Pi/scripts/viewdb.php /home/pi/BirdNET-Pi/scripts/write_config.php
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i 's/http:\/\/birdnetpi.local:8888/http:\/\/birdnetpi.local:8888/g' /home/pi/BirdNET-Pi/scripts/filemanager/filemanager.php
+ '[' -z ']'
++ hostname
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i s/birdnetpi.local/birdnetpi.local/g /home/pi/BirdNET-Pi/homepage/external.html /home/pi/BirdNET-Pi/homepage/footer.html /home/pi/BirdNET-Pi/homepage/index.html /home/pi/BirdNET-Pi/homepage/menu_de.html /home/pi/BirdNET-Pi/homepage/menu.html /home/pi/BirdNET-Pi/homepage/mobile.html /home/pi/BirdNET-Pi/homepage/top_de.html /home/pi/BirdNET-Pi/homepage/top.html
++ hostname
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i s/birdnetpi.local/birdnetpi.local/g /home/pi/BirdNET-Pi/scripts/backup_inprogress.html /home/pi/BirdNET-Pi/scripts/index.html
++ hostname
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i s/birdnetpi.local/birdnetpi.local/g /home/pi/BirdNET-Pi/scripts/backup_inprogress.html /home/pi/BirdNET-Pi/scripts/index.html
++ hostname
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i s/birdnetpi.local/birdnetpi.local/g /home/pi/BirdNET-Pi/scripts/add_to_exclude.php /home/pi/BirdNET-Pi/scripts/add_to_include.php /home/pi/BirdNET-Pi/scripts/adminer-de.php /home/pi/BirdNET-Pi/scripts/adminer-fr.php /home/pi/BirdNET-Pi/scripts/adminer.php /home/pi/BirdNET-Pi/scripts/advanced.php /home/pi/BirdNET-Pi/scripts/birdweather_request.php /home/pi/BirdNET-Pi/scripts/config.php /home/pi/BirdNET-Pi/scripts/del_from_exclude.php /home/pi/BirdNET-Pi/scripts/del_from_include.php /home/pi/BirdNET-Pi/scripts/exclude_list.php /home/pi/BirdNET-Pi/scripts/first_boot.php /home/pi/BirdNET-Pi/scripts/history.php /home/pi/BirdNET-Pi/scripts/include_list.php /home/pi/BirdNET-Pi/scripts/overview.php /home/pi/BirdNET-Pi/scripts/play.php /home/pi/BirdNET-Pi/scripts/service_controls.php /home/pi/BirdNET-Pi/scripts/spectrogram.php /home/pi/BirdNET-Pi/scripts/stats.php /home/pi/BirdNET-Pi/scripts/system_controls.php /home/pi/BirdNET-Pi/scripts/update_inprogress.php /home/pi/BirdNET-Pi/scripts/update_settings_inprogress.php /home/pi/BirdNET-Pi/scripts/viewdb.php /home/pi/BirdNET-Pi/scripts/write_config.php
++ hostname
++ dirname /home/pi/BirdNET-Pi/scripts
+ sudo -upi sed -i s/birdnetpi.local/birdnetpi.local/g /home/pi/BirdNET-Pi/scripts/filemanager/filemanager.php
+ systemctl reload caddy
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to reload 'caddy.service'.
Authenticating as: ,,, (pi)
Password: Failed to reload caddy.service: Connection timed out
See system logs and 'systemctl status caddy.service' for details.
polkit-agent-helper-1: pam_authenticate failed: Authentication failure
+ my_dir=/home/pi/BirdNET-Pi/scripts
+ sudo systemctl stop birdnet_server.service
+ sudo pkill server.py
+ sudo systemctl stop birdnet_recording.service
++ date +%B-%Y/%d-%A
+ sudo rm -rf /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-14:38:37.wav /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-14:38:37.wav.csv /home/pi/BirdSongs/February-2022/24-Thursday/2022-02-24-birdnet-14:38:52.wav
+ services=(web_terminal.service spectrogram_viewer.service pushed_notifications.service livestream.service icecast2.service extraction.service chart_viewer.service birdnet_recording.service birdnet_log.service)
+ for i in "${services[@]}"
+ sudo systemctl restart web_terminal.service
+ for i in "${services[@]}"
+ sudo systemctl restart spectrogram_viewer.service
+ for i in "${services[@]}"
+ sudo systemctl restart pushed_notifications.service
+ for i in "${services[@]}"
+ sudo systemctl restart livestream.service
+ for i in "${services[@]}"
+ sudo systemctl restart icecast2.service
+ for i in "${services[@]}"
+ sudo systemctl restart extraction.service
+ for i in "${services[@]}"
+ sudo systemctl restart chart_viewer.service
+ for i in "${services[@]}"
+ sudo systemctl restart birdnet_recording.service
+ for i in "${services[@]}"
+ sudo systemctl restart birdnet_log.service
+ grep 5050 /dev/fd/63
++ netstat -tulpn
+ sudo systemctl restart birdnet_server.service
+ sleep 30
+ grep 5050 /dev/fd/63
++ netstat -tulpn
+ sudo systemctl restart birdnet_server.service
+ sleep 30
+ grep 5050 /dev/fd/63
++ netstat -tulpn
+ sudo systemctl restart birdnet_server.service
+ sleep 30
+ grep 5050 /dev/fd/63
++ netstat -tulpn
tcp        0      0 127.0.1.1:5050          0.0.0.0:*               LISTEN      914542/python3      
+ sudo systemctl restart birdnet_analysis.service

[mcguirepr89]

Yes, that was as I suspected -- I've fixed the line that prompted for a password (in order to run update_caddyfile.sh in the terminal) 8aad2b5

Thanks for letting me know about what you observed.

[DD4WH]

Made a new install this morning:

• it seems, after setting Audio Channels to 1 and making a "Restart All Services", the Live Stream does not work "404 error"
• Listening to audio recordings works nicely
• Info says: ffmpeg RED, all others GREEN

[mcguirepr89]

Thanks for letting me know!

If you can run the following, it should help me figure out what the issues is:

journalctl -ulivestream --no-pager -n200

And

grep -i icecast /var/log/syslog

And finally, just to be sure ffmpeg installed properly

which ffmpeg

Thank you, Frank! I'm around today, so let me know if you need anything while you're setting up your field device.

[DD4WH]

Thanks, Patrick, here is the ouput of the first command:

And the second command gives a "thousand" lines of output, but basically gives the same error:

I checked my password and it is working and correct (I changed it to "password" in order not to reveal my real password)

[mcguirepr89]

Hmm, those logs only complain that you changed the password for the output :/
Put the Live Stream password back to birdnetpi and restart services and let me know if that fixes things.
I will be removing the ability to set/adjust the icecast2/livestream password in future versions since that password is as safe as it can be since it stays on the loopback network only and cannot be accessed without access to the full machine and since port 8000 would need to be explicitly opened for remote attempts to send data to the stream. (<--- that's a long, crappy sentence! Too lazy to fix it at the moment.)

[DD4WH]

Thanks a lot!!!
That worked fine, now functional again!
Not sure why this did not work yesterday, because I believe I did exactly the same without success . . .

[oliverk71]

Seems, that the menu.html was changed during an update:

[oliverk71]

Repaired it by editing the menu.html

[nilspupils]

Hi Patrick @mcguirepr89 ,
i had this probem of extraction not working with the last system so i did a fresh install yesterday with the 'sqlite' branch.
So it seems the problem is persisting as everything else seems to work but no mp3 files are generated and in the log i get this message:

09:33:45 systemd[1]: birdnet_analysis.service: Succeeded.
09:33:45 systemd[1]: birdnet_analysis.service: Consumed 3.037s CPU time.
09:33:46 env[2364378]: /usr/local/bin/extract_new_birdsounds.sh: Zeile 67: printf: 85.25582600: Ungültige Zahl.
09:33:47 systemd[1]: birdnet_analysis.service: Scheduled restart job, restart counter is at 302.
09:33:47 systemd[1]: Stopped BirdNET Analysis.

(Ungültige Zahl = invalid number)

It is a pretty standard system with a full Bullseye image, german language no big changes.
I will happily provide more details as requested!
Thank you very much!!!
Nils

[DD4WH]

Not sure whether this problem still exists ? I have no problems with the latest newif branch and I use German ...

[nilspupils]

I am not sure either since i am on the the 'newif' branch since then. Will give it a try later!

[mcguirepr89]

Just a note: no need to try the sqlite branch -- and the "fix" isn't working, so no need to even try that! I'm working with Lars on the issue via email and should be able to get to the bottom of things today.

I'll let you all know what I know when I know it :)

[nilspupils]

Side note: The audio stream is working again! Thanks!

[mcguirepr89]

Awesome! I think I've got the fix in place for this:
b382d46

Hopefully that will address any languages that use commas as the locale_decimal