feat(team-provisioner): make forwarding to personal email opt-in

CLAUDE.md

@@ -193,6 +193,7 @@ terraform/state/
 | `scripts/provision-groups.py` | CI: resolve members.yaml + access.yaml, invoke team-provisioner Lambda |
 | `scripts/provision-atlassian.py` | CI: resolve atlassian.yaml to java.no emails, invoke team-provisioner `sync_atlassian` (report-only unless `atlassian_sync_apply`) |
 | `scripts/sync-members.py` | CI: sync approved heroes from Google Sheets into members.yaml |
+| `scripts/gmail-address-map.py` | Manual: members with `forward_to_personal: true` → Gmail recipient address map lines (java.no → personal email), pasted into the "Personlige aliaser" routing rule (tests: `scripts/tests/`) |
 | `scripts/review-plan.py` | CI: LLM plan review via Bedrock (platform CI; also bundled into the plan-review Lambda) |
 | `scripts/invoke-plan-review.sh` | CI: app tf-plan → plan-review Lambda; retries FAILED (service) reviews, says FAILED vs HIGH (tests: `scripts/tests/`) |
 | `scripts/override-help.sh` | CI: what a blocked apply tells the developer (HIGH: request override; FAILED: re-run all jobs) |
@@ -301,7 +302,9 @@ Delegated scopes the platform code requests (Admin console > Security > API cont
 delegation; the field replaces the whole list, so read what is there first and only add, since the
 console may grant scopes not listed here): `admin.directory.group`,
 `admin.directory.group.member`, `admin.directory.user`, `admin.directory.user.alias`, `gmail.send`,
-`gmail.settings.sharing`, `apps.groups.settings` (prefix `https://www.googleapis.com/auth/`).
+`apps.groups.settings` (prefix `https://www.googleapis.com/auth/`). Forwarding to members' personal
+emails is opt-in and a Gmail routing rule set by hand (`scripts/gmail-address-map.py`,
+docs/lambda-functions.md), not an API call.
 Google rejects a token if any requested scope is not delegated, so `apps.groups.settings` is
 requested on a separate token (`_get_google_groups_settings_token`) and team group settings are
 skipped, not failed, when it is missing. The Groups Settings API (`groupssettings.googleapis.com`)

docs/lambda-functions.md

@@ -95,6 +95,23 @@ with no members and the Lambda skips it (no Google, Cognito or Identity Center s
 `scripts/reconcile-groups.py` plans no removals from it. `javazone@java.no`, the JavaZone coordination
 group's list, is manual: only koordineringsgruppa may be in it.
 
+### Forwarding to personal email (opt-in, by hand)
+
+New accounts get a welcome mail at their personal email and no forwarding: mail to a java.no address
+stays in its java.no inbox. A member who wants it forwarded too gets `forward_to_personal: true` in the
+registry's `members.yaml`. Forwarding is a Gmail routing rule, not the Gmail API (which forwards to an
+external address only after the recipient confirms it): Admin console → Apps → Google Workspace → Gmail →
+Routing → "Personlige aliaser" (Email forwarding using recipient address map), with "Also route to
+original destination" ON. The rule has no API. When an opted-in member is added, removed or changes
+`personal_email`, `provision-groups.py` prints and posts a reminder listing the java.no addresses (never
+the personal ones); an admin then replaces the rule's address mappings with the output of
+`scripts/gmail-address-map.py <registry>/groups/members.yaml`, or disables the rule when it prints
+nothing. Registry PR plan comments mark the flag on new and changed members. The map lists primary
+addresses only; check that mail to an opted-in member's `alias` is forwarded too.
+
+The Lambda no longer needs `gmail.settings.sharing` (it set per-user forwarding before). Remove that
+scope from the domain-wide delegation client so the service account key cannot set forwarding.
+
 ### Team permission set (`_ensure_team_permission_set`)
 
 `javabin-team-{team}` attaches the `javabin-team-{team}-guardrails` policy and an inline policy: `Action "*"` on resources tagged `team={team}` (or untagged), denies on platform-managed parameters, a few untagged reads, and `AllowExecOnPlatformCluster`: `ecs:ExecuteCommand` on the shared `javabin-platform` cluster. ECS Exec is authorized against both the cluster and the task, and the task is only allowed through its propagated `team` tag, so members can exec into their own team's tasks only (apps with `compute.exec: true`). Changes reach existing permission sets on the next registry sync, which reprovisions them.
@@ -137,7 +154,7 @@ settings have no visible effect.
 Delegated scopes the platform code requests (client ID 103162295681564397615). The console may grant
 more; read the current list there first and add to it rather than pasting this one:
 `admin.directory.group`, `admin.directory.group.member`, `admin.directory.user`,
-`admin.directory.user.alias`, `gmail.send`, `gmail.settings.sharing`, `apps.groups.settings`
+`admin.directory.user.alias`, `gmail.send`, `apps.groups.settings`
 (all under `https://www.googleapis.com/auth/`).
 
 ### Team GitHub team (`sync_github_team`)

scripts/diff-groups.py

@@ -41,6 +41,10 @@ def diff_members(base_data, pr_data):
             bv, pv = b.get(field, ""), p.get(field, "")
             if bv != pv:
                 diffs.append(f"`{field}`: {bv!r} → {pv!r}")
+        # Absent and false both mean no forwarding.
+        bf, pf = b.get("forward_to_personal") is True, p.get("forward_to_personal") is True
+        if bf != pf:
+            diffs.append(f"`forward_to_personal`: {bf} → {pf} *(update the Gmail address map by hand)*")
         b_groups = set(b.get("memberships") or [])
         p_groups = set(p.get("memberships") or [])
         if b_groups != p_groups:
@@ -154,6 +158,8 @@ def format_markdown(added, removed, changed, group_added, group_removed, group_c
         for m in added:
             name = f"{m.get('firstname', '')} {m.get('lastname', '')}"
             groups = ", ".join(m.get("memberships") or [])
+            if m.get("forward_to_personal") is True:
+                groups += " *(forward_to_personal: update the Gmail address map by hand)*"
             lines.append(f"| {name} | {m.get('javabin_google_email', '')} | {groups} |")
         lines.append("")
 

scripts/gmail-address-map.py

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+"""Print the Gmail recipient address map for members who opt in to forwarding.
+
+Usage: gmail-address-map.py <members.yaml>
+
+Mail to a java.no account stays in its java.no inbox. A member with
+`forward_to_personal: true` in the registry's groups/members.yaml also gets it
+at their `personal_email`, through a Gmail routing rule: Admin console → Apps →
+Google Workspace → Gmail → Routing → "Personlige aliaser" (Email forwarding
+using recipient address map), with "Also route to original destination" ON.
+The rule has no API, so an admin replaces its address mappings with this output
+whenever the opted-in members change, and disables the rule when there are none.
+(Forwarding through the Gmail API needs the recipient to confirm every address.)
+
+Output: one "java.no address, personal address" line per opted-in member, sorted.
+
+Requires: PyYAML
+"""
+
+import argparse
+import sys
+
+
+def address_map(members_data):
+    """Sorted (java.no email, personal email) pairs for members with forward_to_personal: true."""
+    pairs = {
+        m["javabin_google_email"].strip().lower(): m["personal_email"].strip()
+        for m in members_data.get("members") or []
+        if m.get("forward_to_personal") is True
+        and (m.get("javabin_google_email") or "").strip()
+        and (m.get("personal_email") or "").strip()
+    }
+    return sorted(pairs.items())
+
+
+def main():
+    import yaml
+
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument("members_yaml", help="Path to groups/members.yaml")
+    args = parser.parse_args()
+    with open(args.members_yaml) as f:
+        pairs = address_map(yaml.safe_load(f))
+    if not pairs:
+        print("No member has forward_to_personal: true. Disable the routing rule.", file=sys.stderr)
+    for java_no, personal in pairs:
+        sys.stdout.write(f"{java_no}, {personal}\n")
+
+
+if __name__ == "__main__":
+    main()

scripts/provision-groups.py

@@ -135,6 +135,55 @@ def detect_removals(prev_members, curr_members, prev_groups, curr_groups):
     return removed_members, membership_removals, removed_groups
 
 
+def forwarding_changes(prev_members, curr_members):
+    """java.no accounts whose address map entry must change: (turned on, turned off, target changed).
+
+    Each is a sorted list of java.no addresses. "Target changed" means an opted-in member's
+    personal_email changed, so the rule would keep forwarding to the old address.
+    """
+    def opted_in(data):
+        return {
+            m["javabin_google_email"].strip().lower(): (m.get("personal_email") or "").strip().lower()
+            for m in (data.get("members") or [])
+            if (m.get("javabin_google_email") or "").strip() and m.get("forward_to_personal") is True
+        }
+    prev, curr = opted_in(prev_members), opted_in(curr_members)
+    changed = sorted(e for e in prev.keys() & curr.keys() if prev[e] != curr[e])
+    return sorted(curr.keys() - prev.keys()), sorted(prev.keys() - curr.keys()), changed
+
+
+def post_forwarding_reminder(turned_on, turned_off, target_changed=()):
+    """Tell the admin to update the Gmail recipient address map (it has no API).
+
+    Lists java.no addresses only; personal emails stay out of Slack.
+    """
+    lines = [":envelope: *Forwarding to personal email changed* — update the Gmail routing rule by hand", ""]
+    if turned_on:
+        lines.append("*Turned on:* " + ", ".join(turned_on))
+    if turned_off:
+        lines.append("*Turned off:* " + ", ".join(turned_off))
+    if target_changed:
+        lines.append("*Personal email changed:* " + ", ".join(target_changed))
+    lines.append("")
+    lines.append(
+        "Run `scripts/gmail-address-map.py groups/members.yaml` (javaBin/platform) and replace the "
+        "address mappings of \"Personlige aliaser\" (Admin console → Gmail → Routing) with its output, "
+        "\"Also route to original destination\" ON. No lines: disable the rule."
+    )
+    text = "\n".join(lines)
+    print(text)
+    webhook_url = os.environ.get("SLACK_WEBHOOK_URL")
+    if not webhook_url:
+        return
+    req = urllib.request.Request(
+        webhook_url, data=json.dumps({"text": text}).encode(), headers={"Content-Type": "application/json"},
+    )
+    try:
+        urllib.request.urlopen(req)
+    except Exception as e:
+        print(f"Warning: Slack notification failed: {e}", file=sys.stderr)
+
+
 def post_removal_warning(removed_members, membership_removals, removed_groups):
     """Post a removal warning to Slack."""
     webhook_url = os.environ.get("SLACK_WEBHOOK_URL")
@@ -225,6 +274,10 @@ def main():
         else:
             print("\nNo removals detected")
 
+        changes = forwarding_changes(prev_members, members_data)
+        if any(changes):
+            post_forwarding_reminder(*changes)
+
     # Pass CI context so the Lambda can attribute the sync in Slack
     ci_context = {
         k: v for k, v in {

scripts/sort-members.py

@@ -83,6 +83,8 @@ def format_yaml(sections):
                 lines.append(f"    javabin_google_email: {m['javabin_google_email']}")
                 alias = m.get("alias") or ""
                 lines.append(f"    alias: {alias}" if alias else "    alias:")
+                if m.get("forward_to_personal") is True:
+                    lines.append("    forward_to_personal: true")
                 memberships = m.get("memberships", [])
                 if memberships:
                     lines.append(f"    memberships: [{', '.join(memberships)}]")

scripts/sync-members.py

@@ -242,6 +242,8 @@ def format_yaml(members):
             lines.append(f"    javabin_google_email: {m['javabin_google_email']}")
             alias = m.get("alias") or ""
             lines.append(f"    alias: {alias}" if alias else "    alias:")
+            if m.get("forward_to_personal") is True:
+                lines.append("    forward_to_personal: true")
             memberships = m.get("memberships", [])
             if memberships:
                 lines.append(f"    memberships: [{', '.join(memberships)}]")
@@ -284,8 +286,10 @@ def merge_heroes(existing, new_entries):
         key = entry["personal_email"].lower()
         if key in merged:
             old = merged[key]
-            # Preserve alias from existing entry
+            # Preserve alias and forwarding opt-in (set in the registry, never by the sheet)
             entry["alias"] = old.get("alias") or entry.get("alias") or ""
+            if old.get("forward_to_personal") is True:
+                entry["forward_to_personal"] = True
             # Update memberships from sheet (source of truth for group affiliations)
             merged[key] = entry
             updated += 1

scripts/tests/test_forwarding.py

@@ -0,0 +1,151 @@
+"""Forwarding to personal email is opt-in (`forward_to_personal: true` in members.yaml).
+
+Seams: the scripts' public functions and `provision-groups.py main()`, with the Lambda
+invocation and the Slack webhook (urllib) patched out.
+
+Run: python3 -m unittest discover -s scripts/tests
+"""
+
+import contextlib
+import importlib.util
+import io
+import json
+import os
+import sys
+import tempfile
+import unittest
+from unittest import mock
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+
+def load(name, filename):
+    spec = importlib.util.spec_from_file_location(name, os.path.join(HERE, "..", filename))
+    mod = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(mod)
+    return mod
+
+
+def member(first, forward=None, **extra):
+    m = {"firstname": first, "lastname": "Nordmann", "personal_email": f"{first.lower()}@gmail.com",
+         "javabin_google_email": f"{first.lower()}.nordmann@java.no", "memberships": ["helter"], **extra}
+    if forward is not None:
+        m["forward_to_personal"] = forward
+    return m
+
+
+MEMBERS = {"members": [member("Ola", True), member("Kari"), member("Per", False), member("Anne", "yes")]}
+
+
+class AddressMapTest(unittest.TestCase):
+    def setUp(self):
+        self.gam = load("gmail_address_map", "gmail-address-map.py")
+
+    def test_only_opted_in_members(self):
+        self.assertEqual(self.gam.address_map(MEMBERS), [("ola.nordmann@java.no", "ola@gmail.com")])
+
+    def test_needs_both_addresses(self):
+        data = {"members": [member("Ola", True, personal_email=""), member("Kari", True, javabin_google_email=" ")]}
+        self.assertEqual(self.gam.address_map(data), [])
+
+
+class ProvisionGroupsForwardingTest(unittest.TestCase):
+    def setUp(self):
+        try:
+            self.pg = load("provision_groups_fwd", "provision-groups.py")
+        except ModuleNotFoundError as e:  # PyYAML is on the CI runner
+            self.skipTest(str(e))
+
+    def test_changes(self):
+        now = {"members": [member("Ola"), member("Kari", True), member("Per", False)]}
+        self.assertEqual(self.pg.forwarding_changes(MEMBERS, now),
+                         (["kari.nordmann@java.no"], ["ola.nordmann@java.no"], []))
+        self.assertEqual(self.pg.forwarding_changes(MEMBERS, MEMBERS), ([], [], []))
+
+    def test_personal_email_change_of_opted_in_member(self):
+        now = {"members": [member("Ola", True, personal_email="ola@new.example")]}
+        self.assertEqual(self.pg.forwarding_changes(MEMBERS, now), ([], [], ["ola.nordmann@java.no"]))
+
+    def run_main(self, prev, curr):
+        """Run main() on YAML files; returns the Slack payloads it posted."""
+        import yaml
+        with tempfile.TemporaryDirectory() as d:
+            paths = {}
+            for name, data in {"groups": {"groups": []}, "prev": prev, "curr": curr}.items():
+                paths[name] = os.path.join(d, f"{name}.yaml")
+                with open(paths[name], "w") as f:
+                    yaml.safe_dump(data, f)
+            argv = ["provision-groups.py", paths["groups"], paths["curr"], "--prev-members", paths["prev"]]
+            with mock.patch.object(sys, "argv", argv), \
+                    mock.patch.dict(os.environ, {"SLACK_WEBHOOK_URL": "https://hooks.example/x"}), \
+                    mock.patch.object(self.pg, "invoke_lambda"), \
+                    mock.patch.object(self.pg.urllib.request, "urlopen") as urlopen, \
+                    contextlib.redirect_stdout(io.StringIO()):
+                self.pg.main()
+        return [json.loads(c.args[0].data)["text"] for c in urlopen.call_args_list]
+
+    def test_main_posts_reminder_without_personal_emails(self):
+        now = {"members": [member("Ola", True, personal_email="ola@new.example"), member("Kari", True),
+                           member("Per", False), member("Anne", "yes")]}
+        texts = [t for t in self.run_main(MEMBERS, now) if "Forwarding to personal email" in t]
+        self.assertEqual(len(texts), 1)
+        self.assertIn("*Turned on:* kari.nordmann@java.no", texts[0])
+        self.assertIn("*Personal email changed:* ola.nordmann@java.no", texts[0])
+        for m in MEMBERS["members"] + now["members"]:
+            self.assertNotIn(m["personal_email"], texts[0])
+
+    def test_main_posts_nothing_when_opt_ins_unchanged(self):
+        self.assertEqual(self.run_main(MEMBERS, MEMBERS), [])
+
+    def test_member_details_carry_no_forwarding(self):
+        # The Lambda never sets up forwarding; nothing in the payload asks for it.
+        details = self.pg.build_member_details(MEMBERS)
+        self.assertTrue(all("forward_to_personal" not in d for d in details))
+
+
+class DiffGroupsForwardingTest(unittest.TestCase):
+    def setUp(self):
+        try:
+            self.dg = load("diff_groups_fwd", "diff-groups.py")
+        except ModuleNotFoundError as e:
+            self.skipTest(str(e))
+
+    def test_flag_change_shows_in_plan(self):
+        _, _, changed = self.dg.diff_members({"members": [member("Ola")]}, {"members": [member("Ola", True)]})
+        self.assertEqual(len(changed[0]["diffs"]), 1)
+        self.assertIn("`forward_to_personal`: False → True", changed[0]["diffs"][0])
+        # an explicit false is the same as no flag
+        self.assertEqual(self.dg.diff_members({"members": [member("Ola", False)]}, {"members": [member("Ola")]})[2], [])
+
+    def test_new_member_opt_in_shows_in_plan(self):
+        added, _, _ = self.dg.diff_members({"members": []}, {"members": [member("Ola", True)]})
+        md = self.dg.format_markdown(added, [], [], [], [], [], [], [])
+        self.assertIn("forward_to_personal", md)
+
+
+class SerializersKeepFlagTest(unittest.TestCase):
+    """The sort bot and the sheet sync rewrite members.yaml; they must keep the opt-in."""
+
+    def test_sort_members_keeps_flag(self):
+        try:
+            sort = load("sort_members_fwd", "sort-members.py")
+        except ModuleNotFoundError as e:
+            self.skipTest(str(e))
+        out = sort.format_yaml(sort.sort_members(MEMBERS["members"]))
+        self.assertEqual(out.count("forward_to_personal: true"), 1)
+        ola = out[out.index("firstname: Ola"):]
+        self.assertLess(ola.index("forward_to_personal: true"), ola.index("memberships:"))
+
+    def test_sync_members_preserves_flag_from_registry(self):
+        sm = load("sync_members_fwd", "sync-members.py")
+        existing = {m["personal_email"]: dict(m) for m in MEMBERS["members"]}
+        fresh = [{**member("Ola"), "memberships": ["helter", "kodesmia"]}, member("Kari")]
+        merged, _, _ = sm.merge_heroes(existing, fresh)
+        by_first = {m["firstname"]: m for m in merged}
+        self.assertIs(by_first["Ola"].get("forward_to_personal"), True)
+        self.assertNotIn("forward_to_personal", by_first["Kari"])
+        self.assertEqual(sm.format_yaml(merged).count("forward_to_personal: true"), 1)
+
+
+if __name__ == "__main__":
+    unittest.main()