• Bugzilla
• Setup

Bugzilla Setup

Introduction

FreeBSD relies on Bugzilla for problem reports and suggested patches.

There are two installations: kenobi (the production machine), and flytrap (the development machine).

• As of 20250114, both of our installations are at 5.0.4.1. This is downrev; see upstream's roadmap for the gory details.

• However, both installations have their own FreeBSD customizations and patches, meaning that updating is non-trivial. This page will attempt to quantify them.

Setting Up The Bugzilla Jail Itself

• production:
  • bugzilla installed from pkg by clusteradm
• development:

  • bugzilla installed from cd ${PORTSDIR}/devel/bugzilla(50|52}; make install

  • see README.md
• both:
  • postgres and apache installed from pkg by clusteradm@
  • postgres and apache configured by clusteradm@
  • bugzilla config in apache done by clusteradm@
  • external proxy config done by clusteradm@

Ensure that /etc/localtime exists and points to the correct timezone. Otherwise, the DateTime module will fail to work, resulting in a broken Bugzilla installation.

Clusteradm Package Customizations

for Bugzilla 5.2, s/50/52/ .

devel_bugzilla50_SET+=EXPORT_IMPORT
devel_bugzilla50_SET+=GRAPHVIZ
devel_bugzilla50_SET+=INBOUND_EMAIL
devel_bugzilla50_SET+=JSONRPC
devel_bugzilla50_SET+=LDAP
devel_bugzilla50_SET+=MAIL_QUEUEING
devel_bugzilla50_SET+=MIME_SNIFF
devel_bugzilla50_SET+=MOREBUGSURL
devel_bugzilla50_SET+=MOVE_BUGZ
devel_bugzilla50_SET+=PGSQL
devel_bugzilla50_SET+=REST
devel_bugzilla50_SET+=XMLRPC

See also: @FreeBSD.org patches to Bugzilla/Auth/Verify/LDAP.pm; token.cgi.

NB: MOREBUGSURL is a typo. TBD.

Bugzilla Site Customizations

Do the first-level check. It will probably throw errors.

  # cd /usr/local/www/bugzilla
  # chmod 775 .
  # chgrp -R bugzilla .
  # ./checksetup.pl

Edit /usr/local/www/bugzilla/localconfig (substituting clusteradm-supplied values for HOST, DB*, and SECRET):

  $create_htaccess = 1;
  $webservergroup = 'bugzilla';
  $use_suexec = 0;
  $db_driver = 'Pg';
  $db_host = '<HOST>';
  $db_name ='<DBNAME>';
  $db_user = '<DBUSER>';
  $db_pass = '<DBPASS>';
  $db_port = 0;
  $db_sock = '';
  $db_check = 1;
  $index_html = 0;
  $cvsbin = '';
  $interdiffbin = '/usr/local/bin/interdiff';
  $diffpath = '/usr/bin';
  $site_wide_secret = '<SECRET>';

Ensure that the following have been done:

  sysrc postgresql_enable="yes"
  sysrc postgresql_user="pgsql"
  sysrc postgresql_data="/usr/local/pgsql/data"
  service postgresql initdb

Create db user and database according to the installation guide:

  # su - pgsql
  # createuser -d postgres -dRSP <DBUSER>

run ./checksetup.pl again

Add <bugmeister AT FreeBSD DOT org> as admin account.

Replace <DBNAME> database with the dump

service apache24 onestart

Login to the bugzilla instance

Go to Administration and make the following changes.

note: internally the below is stored as data/params.json. As stored, the file contents are sorted. Plus, "0" is used for "Off", "1" for "On".

Required Settings:

  urlbase: setup the correct urlbase (e.g. https://bugs.freebsd.org/bugzilla/, https://bugstest.freebsd.org/bugzilla/)
  ssl_redirect: On (production), Off (development)
  sslbase: setup the correct sslbase (same as urlbase)
  cookiepath: /bugzilla/

General:

  maintainer: bugmeister@FreeBSD.org
  utf8: On
  announcehtml: see prod installation
  upgrade_notification: latest_stable_release (production), disabled (development)

Administrative Policies:

  allowbugdeletion: Off
  allowemailchange: On
  allowuserdeletion: Off

User Authentication:

  auth_env_*: keep all three empty
  user_info_class: CGI
  user_verify_class: FreeBSD,DB
  rememberlogin: defaulton
  requirelogin: Off
  emailregexp: [\\w\\.\\+\\-=]+@[\\w\\.\\-]+\\.[\\w\\-]+$
  emailregexpdesc: A legal address must contain exactly one '@', and at least one '.' after the @.
  emailsuffix: empty
  createemailregexp: .*   # !!!NOTE: leave this blank on the test system to disable external registration!!!
  password_complexity: no_constraints

Attachments:

  allow_attachment_display: On
  attachment_base: https://bz-attachments.freebsd.org/ (production), empty (development)
  allow_attachment_deletion: On (production), Off (development)
  maxattachmentsize: 1000
  maxlocalattachment: 0

Bug Change Policies:

  duplicate_or_move_bug_status: Closed (requires changes to the bug status)
  letsubmitterchoosepriority: Off
  letsubmitterchoosemilestone: Off
  musthavemilestoneonaccept: Off
  commentonchange_resolution: Off
  commentonduplicate: Off
  noresolveonopenblockers: On

Bug Fields:

  useclassification: Off
  usetargetmilestone: Off
  useqacontact: Off
  usestatuswhiteboard: Off
  use_see_also: On
  defaultpriority: ---
  defaultseverity: Affects Only Me (requires changes to the severity)
  defaultplatform: Any (requires changes to the plaform)
  defaultopsys: Any (requires changes to the opsys)

Dependency Graphs:

  webdotbase: /usr/local/bin/dot

Group Security:

  makeproductgroups: Off
  chartgroup: admin
  insidergroup: empty
  timetrackinggroup: empty
  querysharegroup: editbugs
  debug_group: admin
  usevisibilitygroups: On
  strict_isolation: Off

LDAP:

  LDAPserver: ldap1.nyi.freebsd.org,ldap2.nyi.freebsd.org
  LDAPstarttls: On
  LDAPbinddn: empty
  LDAPBaseDN: ou=users,dc=freebsd,dc=org
  LDAPuidattribute: uid
  LDAPmailattribute: empty
  LDAPfilter: empty

RADIUS:

 leave all empty 

Email:

  mail_delivery_method: Sendmail (production), Test (development)
  mailfrom: bugzilla-noreply@freebsd.org (production), bugzilla-noreply-test@freebsd.org (development)
  use_mailer_queue: Off
  smtpserver: localhost
  smtp_username: empty
  smtp_password: empty
  smtp_ssl: Off
  smtp_debug: Off
  whinedays: 7
  globalwatchers: empty

Patch Viewer:

 leave all empty 

Query Defaults:

  quip_list_entry_control: moderated (production), closed (development)
  mostfreqthreshold: 2 (XXX not found 20250114)
  mybugstemplate: buglist.cgi?resolution=---&emailassigned_to1=1&emailreporter1=1&emailtype1=exact&email1=%userid%
  defaultquery: emailassigned_to1=1&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailqa_contact2=1&emaillongdesc3=1&order=Importance&long_desc_type=substring
  search_allow_no_criteria: On
  default_search_limit: 500
  max_search_results 12500

MCL 20231130 max_search_results being 10000 has been too low for some time. I have raised it.

Shadow Database:

 leave all empty 

User Matching:

  usemenuforusers: Off
  ajax_user_autocompletion: On
  maxusermatches: 50
  confirmuniqueusermatch: Off

Advanced:

  cookiedomain: host part of the urlbase (e.g. bugs.freebsd.org, bugstest.freebsd.org)
  inbound_proxies: 8.8.178.110,2001:1900:2254:206a::50:0,127.0.1.11
  proxy_url: empty
  strict_transport_security: this_domain_only (production), off (development)

Installed Bugzilla Addons

It appears as of 20231201 we do not use any of these addons.

• current list of upstream's supported addons

Installed Bugzilla Extensions

Introduction

For reference, see the somewhat out-of-date Bugzilla Extension tutorial.

Also see the current list of upstream's supported Extensions (claimed)

The extension files are deployed in:

• /usr/local/www/bugzilla/extensions/<extension>

• /usr/local/www/bugzilla/data/template/extensions/<extension>/template/en/default/pages/

Deployment Matrix

Unless otherwise specified, the upstream for our local patches is FreeBSD Bugzilla github. MarkLinimon uses the linimon Bugzilla git repo as a staging area of 20241231.

Installation

XXX TODO in progress

field.js fix submitted upstream but also now in files/XXX TODO

cd /var/www/local/bugzilla/extensions
git clone https://github.com/bayoteers/BayotBase.git
git clone https://github.com/bayoteers/BugViewPlus.git
NOTYET git clone https://github.com/bayoteers/SubscribeField.git
cd ../..
patch < /home/linimon/clusteradm.patch

Implementation Details

If you want to find out which pages can be displayed (whether or not they appear in the Toolbar), do the following:

• grep page.cgi /usr/local/www/bugzilla/extensions/*/template/en/default/pages/*.tmpl

Note: the SVNLinks title is obsolete -- the Git code now lives in there as well. So don't delete/disable it!

Local Bugzilla Patches

There are several types of patches:

• porting patches (e.g. s@/usr/bin/perl@/usr/local/bin/perl@)

  • in theory, these are all incorporated in the FreeBSD bugzilla ports for 5.0 and 5.2,

    • and thus you do not need to do anything.
• rollup patches from upstream

  • whine.pl

• clusteradm patches. These only affect installations actually deployed on @FreeBSD.org machines.

  • affected: Auth/Verify/LDAP.pm, token.cgi

• localizations of template files
  • Bugzilla is engineered to use each file automatically if they exist.

  • The localizations live in ./template/en/custom/

  • These override the default fallbacks which live in ./template/en/default/

Deployment

  # git clone https://github.com/freebsd/bugzilla
  # cp -rfv bugzilla/* /usr/local/www/bugzilla
  # /usr/local/www/bugzilla/checksetup.pl
  # chown -R bugzilla /usr/local/www/bugzilla/data/mining
  # chmod g+rx /usr/local/www/bugzilla/weeklyreminder.pl

AutoAssigner support:

document git replacement TODO 202050103

  # mkdir /home/ports
  # chown bugzilla:bugzilla /home/ports
  # su - bugzilla
  # cd /home/ports
  # svn checkout svn://svn.freebsd.org/ports/head .

Crontab for bugzilla (production only):

You need to call out perl explicitly because otherwise checksetup.pl resets the group permission of *.pl, and thus bugzilla crontab cannot run them! in progress 20250103

  @reboot         /usr/local/bin/fetchmail -d 60 >/dev/null 2>&1
  */15 * * * *    /usr/local/bin/fetchmail -d 60 >/dev/null 2>&1

  MAILTO=bugmeister@freebsd.org
  5 0 * * *       cd /usr/local/www/bugzilla; perl ./collectstats.pl 2>&1 | grep -v deprecated
  5 3 * * *       cd /usr/local/www/bugzilla; perl ./sync_freebsd_committers.pl 2>&1 | grep -v deprecated
  55 0 * * *      cd /usr/local/www/bugzilla; perl ./whineatnews.pl 2>&1 | grep -v deprecated
  */15 * * * *    cd /usr/local/www/bugzilla; perl ./whine.pl 2>&1 | grep -v deprecated | grep -v Insecure | grep -v uninitialized
  */15 * * * *    cd /home/ports && /usr/local/bin/git pull >/dev/null 2>&1
  0 21 * * 0      cd /usr/local/www/bugzilla; perl ./weeklyreminder.pl 2>&1 | grep -v deprecated

Upgrades

Document upgrade steps. Will reorganize/merge this section with Deployment section to the extent that they may overlap after we have described an upgrade process completely here in standalone form

Database Backups

FreeBSD Bugzilla Extensions come with a small database backup script, dbbackup.sh. It expects a backup directory /var/backups/bugzilla with enough space to store a minimum of 15 backups (one per day, 14 days). Backups older than 14 days are deleted automatically, once a new backup was created.

Change the DBNAME and DBUSER entries within dbbackup.sh to point to the correct database.

Crontab for root (database backups):

  MAILTO=clusteradm-alerts@freebsd.org,bugmeister@FreeBSD.org
  0 3 * * *       cd /usr/local/www/bugzilla/contrib && env PATH="${PATH}:/usr/local/bin" sh dbbackup.sh

Quirks

These are not quirks! They are bugs w/rt permissions with things run from bugzilla crontab! But as of 20250114, they are believed fixed.

• cron weeklyreminder.pl Can't open perl script "./weeklyreminder.pl": Permission denied

  • Fix: # chmod g+rx /usr/local/www/bugzilla/weeklyreminder.pl

• cron collectstats.pl Unable to open the chart datafile ./data/mining/0.

  • Fix: # chown -R bugzilla /usr/local/www/bugzilla/data/mining

Email

This documents some obsolete garbage that is still live on kenobi as of 20240615.

There are two email addresses handled in /home/bugzilla/.fetchmailrc.

username bugdrop

freebsd-gnats-submit@freebsd.org is still live as of 20240615. MarkLinimon considers this a bug.

• input is snarfed via pop3 from freefall into /home/bugzilla/q on kenobi in /home/bugzilla/recv.sh.

• in almost all cases the content is crap. Each piece of crap creates an email "NOT A PR" or similar to bugmeister@.

• in a few miserable cases "freebsd-gnats-submit@freebsd.org" is in the Cc: of antique PRs. This

  • should never have happened in the first place.

Suggestion: orbital railgun.

username bzapi

I'm not sure about this one.

• input is snarfed via pop3 from freefall into /usr/local/www/bugzilla/email_in.pl. This is part of the standard Bugzilla distribution.

• But ... what is it doing?