A supplier field trial for an RF drone detection system needs comparable target opportunities, independent ground truth, and records buyers can audit. Use it to shortlist credible candidates before award; acceptance of the installed system comes later.
A polished demonstration may show a convincing alert without proving what caused it, whether the aircraft was identified correctly, or how often the detector missed a target. Define the outputs the operation needs first. Then hold the test conditions and scoring rules constant across suppliers.
1. What Must an RF Drone Detection Supplier Field Trial Prove?

Turn the operational requirement into specific outputs before discussing detection range. An alert, an identity, a position, and a track answer different questions.
Score detections, classifications, identifications, positions, and tracks as distinct outputs. Classification may assign a category, while identification may name a model, protocol, or individual identifier. An alert does not establish identity, and a single position is not a continuing track. Location records must distinguish the aircraft, its controller, and a reported takeoff point; none automatically establishes a person’s current position.
Passive RF detection needs a relevant signal from the aircraft or associated equipment that the detector can receive. Record RF-silent and unsupported autonomous targets as explicit coverage cases. If neither source emits such a signal, a passive RF-only system cannot detect that target. Where this coverage is mandatory, evaluate complementary sensing or record the RF-only capability gap; do not quietly exclude the target from the procurement decision.
The procurement team should separate mandatory capabilities from useful extras. For each required output, name the target categories, operating area, response task, and available staff. The engineering team can then define a measurable objective and the evidence needed to judge it. “Provide an alert during the agreed target exposure window” is testable; “detect drones reliably” is not yet a scoring rule.
Before inviting suppliers, decide whether RF, radar, or multi-sensor detection fits the mandatory target list. That decision determines which equipment belongs in the field trial.
2. How Do You Build a Common Test Matrix for All Suppliers?

Set the same operational tasks and scoring definitions for every supplier. Allow different approved mounts where necessary, but record antenna position, height, orientation, configuration, and site conditions that could affect results.
Give suppliers a commissioning period, then freeze each tested configuration. Before scoring, fix target-opportunity counts for each required target, protocol, and scenario, plus no-target monitoring hours under defined RF conditions. If the team uses sequential testing, approve a statistically valid uncertainty method and stopping rule in advance. Group related passes by flight, time block, and RF condition; repeated passes are not automatically independent samples. Balance supplier order and time slots. Never add scored runs selectively after reviewing early results.
This matrix is a planning template. Fill in routes, distances, heights, repetitions, and scoring windows for the agreed targets and protocols before issuing the trial schedule.
Common Supplier Trial Matrix
| Scenario / run identifier | Target and protocol | Deployment and flight conditions | Required output | Repetition and scoring rules | Ground-truth and log sources |
|---|---|---|---|---|---|
| Baseline approach; one ID per repeat | Named drone, controller, protocol, and mode | Route, altitude, direction, coverage zone, and antenna setup | Required alert, identity, position, or track | Frozen repeats and event-matching rules | Independent flight/observer truth and detector export |
| Changed approach or obstruction; separate run IDs | Same agreed target/protocol combinations | Direction, terrain, or line-of-sight change; other conditions logged | Same required outputs under changed geometry | Score separately from baseline | Position truth, obstruction notes, and system logs |
| Busy RF environment; separate run IDs | Named target and operating mode | Recorded ambient RF activity and installation setup | Required outputs amid background signals | Repeats under documented RF conditions; no unqualified pooling | Target truth, RF-background record, and event logs |
| Concurrent targets, when required; separate run IDs | Named target group, protocols, and modes | Simultaneous activity, target separation, and RF background | Per-target alerts, identities, or tracks | Count misses, merged targets, and identity switches | Independent time-aligned truth for each target and system logs |
| No-target monitoring; one ID per window | No planned target; log other observed activity | Agreed area, monitoring hours, and RF background | All alerts for later adjudication | Count alarm episodes over valid online time; retain unresolved events | Independent observation, availability, and alert logs |
| Controller powered before takeoff; separate event IDs | Named drone, controller state, and protocol | Power-on, ground activity, and takeoff recorded separately | RF-presence output versus airborne-target claim | Apply the agreed output rule; do not assume an RF alert is false | Controller/activity truth and timestamped alerts |
Record unsupported target categories before testing. If one is mandatory, keep that gap in the decision record even when the supplier performs well on supported targets.
Approve valid-run rules, flight safety, site permissions, and lawful RF and data collection before testing. For US trials, seek qualified legal review of RF monitoring, decoding, and data retention. At airports, check FAA coordination requirements before deployment. Check the detector’s actual receive and transmit functions, including setup, update, or calibration modes; a passive label alone does not establish that every mode is non-transmitting. This is a detection trial and does not require interference or mitigation exercises.
3. How Do You Establish Independent Ground Truth?

Ground truth is the independent reference for what actually happened, when it happened, and where. It must not come solely from the detector being evaluated.
Use authorized aircraft flight records, suitable independent position references, and trained observers as the task requires. Record the reference method’s uncertainty and any gaps. An observer may confirm takeoff and landing without providing the position accuracy needed to assess a supplier’s location claim.
Check clock alignment before and after each scored test block across the truth source, detector, recording system, and observers. Record any offset or drift that could affect event matching or latency. Set the time basis and allowed matching window. For spatial matching, specify whether a target opportunity starts when the aircraft enters the agreed coverage zone.
Match each scored system output to the correct target and event. The matching rule should resolve duplicate alerts and ambiguous associations without relying on the vendor operator’s recollection.
Consider a controller that starts transmitting before takeoff. An early RF alert may correctly indicate radio activity. It does not, by itself, show that an aircraft was airborne. The truth record must distinguish those events before the evaluator scores the output.
If clock uncertainty, missing truth, or position-reference error prevents a defensible judgment, pause scoring for the affected metric. A supplier’s own map marker cannot independently validate its location accuracy.
4. How Should Operator Roles and Blind Trials Be Controlled?

Assign responsibility before the trial begins. The trial director controls the approved plan and records deviations. The pilot and safety team control flight safety. An independent recorder maintains ground truth, the vendor operator runs the detector, and the scoring/data custodian preserves records and applies the agreed rules.
If one person fills several roles, document the overlap and independent checks. The vendor must not be the sole recorder of target activity or the judge of its own alerts.
Before scoring, record each device identity, software and firmware versions, protocol-library version if applicable, and active configuration.
For blind scored runs, withhold exact event timing and sequence from the vendor operator where practical. Keep the pilot and safety team fully informed of the approved flight plan. Blinding must never remove safety information or prevent an abort.
Record who operated the system, what assistance they received, and every adjustment during scored activity. Separate supplier-assisted operation from buyer-operated trials if both matter to procurement. Without that distinction, supplier coaching may inflate the apparent system performance.
Apply the agreed deviation rule to any scored run compromised by a cue, intervention, or configuration change. Keep that run separate from unchanged blind trials.
5. Which RF Detection Metrics Can Actually Be Compared?

Compare metrics only when their definitions, eligible events, target conditions, and evidence align. An attractive number with a different denominator answers a different question.
Agree the definitions below before suppliers see the scored results.
Performance Metric Definitions and Comparability
| Metric | Eligible events and calculation | Time, space, and target scope | Required evidence | Not directly comparable when |
|---|---|---|---|---|
| End-to-end detection success rate | Matched detections / all eligible opportunities, including detector outages | Target/protocol, scenario, zone, and window | Independent truth, matching rules, detection and availability logs | Target sets, misses, or exposure windows differ |
| Detection rate while operational | Matched detections / eligible opportunities while all required functions are ready | Same target conditions and frozen definition of ready | Truth, detection events, and availability timeline | Operating state or eligible opportunities differ |
| Detector availability | Time all required detection and alert functions are ready / required monitoring time; pre-agreed exclusions only | Agreed duty periods and ready-state definition | Health logs and independent monitoring timeline | Device-on is treated as ready, or outages are hidden |
| Confirmed false-alarm rate; unresolved alerts | Confirmed false-alarm episodes / valid online hours; unresolved events and alarm duration separately | Scored output type, observed area, and RF conditions | Alert records, independent observations, and availability logs | Alert grouping, truth coverage, exclusions, or unknowns differ |
| Detection latency | First qualifying output time minus agreed reference-event time; misses separately | Same start event, output stage, and target scenario | Aligned truth/output timestamps and clock-offset record | Start events differ or pre-existing alerts are mishandled |
| Identification correctness and availability | Freeze one ID decision per eligible opportunity; correctness = correct issued decisions / all scored issued decisions; availability = opportunities with an in-window ID at the required level / all eligible identity opportunities. Report wrong, conflicting, missing, and corrected outcomes separately | Same identity level, target/protocol set, and counting unit | Known identity and complete identity-output history | Corrected IDs hide earlier errors, or ID levels differ |
| Location error and availability | Error = agreed 2D/3D distance (m) between each valid system position and time-matched independent truth, with distribution summaries and sample counts; availability = eligible target time with valid, sufficiently recent positions / all eligible target time. Report missing and stale positions separately | Same located object, coordinate basis, 2D/3D measure, and output-age limit | Independent position truth and uncertainty; system position history | Summary statistics differ, or aircraft and controller locations are mixed |
| Track continuity | Valid associated-track time / eligible target time; also gaps and identity switches | Same association, update, and gap rules | Time-aligned target truth and track history | Gap or target-association rules differ |
Detection Probability and False Alarm Measurement
Define one detection opportunity before counting results. It may be a passage through a specified zone. Several screen updates during that passage still represent one opportunity.
Report each target, protocol, and scenario before any aggregate, along with counts and the agreed uncertainty method. Nine detections in ten independent opportunities and ninety in one hundred both yield 90%; the larger independent sample supports a tighter uncertainty range. Repeated flights in the same conditions may not provide that independence. Neither result proves performance on untested targets.
The end-to-end rate includes eligible opportunities missed during outages. Report it with detection rate while operational and detector availability. A detector that performs well only when online may still leave unacceptable monitoring gaps.
Define false alarms against the claimed output. An RF-presence indication, airborne-drone alert, and model identification have different truth conditions. Freeze the episode-grouping rule and record both count and duration. Valid monitoring hours require known detector availability and enough independent observation to judge alerts. A window with no planned target may still contain other aircraft or RF activity.
Keep alerts without adequate truth as unresolved. Report confirmed false-alarm episodes per valid online hour, unresolved counts, total alert burden, and hours excluded for truth gaps or downtime. A confirmed-only rate understates the possible burden when unresolved alerts may be false. Hold the comparison if those unknowns could change the ranking.
Zero confirmed false alarms during a short observation does not establish a zero long-term false-alarm rate. Report valid observed hours and downtime; never use denominator exclusions to hide system unavailability.
Detection Latency, Identity, and Location Accuracy
Choose the latency endpoint the operation needs: first system event, operator-visible alert, or another specified output. Handle alerts already active at the reference event under a frozen rule; they must not create misleading zero or negative latency. Show misses beside latency for successful detections.
Keep wrong and conflicting identity labels, even when the system corrects them later. For location, agree on a summary such as the median and a high percentile in advance; also report sample counts, stale outputs, and reference uncertainty. A correct model label is not a unique aircraft ID; accurate positions from a few moments do not show continuous availability.
Compare metrics only where their definitions and conditions align. If only some evidence qualifies, compare that portion and state the remaining gap. Hold the affected ranking when event definitions or reference data remain unresolved.
6. What Logs and Data Rights Must Buyers Secure?

Agree the evidence package and buyer access rights before the trial. A dashboard demonstration is not enough if the evaluation team cannot later reconstruct what happened.
Each run needs a traceable connection between the target, truth record, and tested configuration. Request device identities, software/firmware and relevant protocol-library versions, active settings, run IDs, timestamps, alerts, identities, positions or tracks, errors, and operator changes. Record which requested outputs the system does not provide.
Check a sample export before scoring. The evaluation team should be able to interpret the format, field definitions, time basis, coordinate system, and missing-value codes. It should also be able to match exported events to the independent truth record.
Define who owns each dataset and what the buyer may retain, analyze, and share with an authorized evaluator. Agree export delivery, retention periods, access after the trial, and confidentiality controls for sensitive location and operator information. Keep original exports intact and document any later corrections or processing.
Request enough evidence to audit the result. Raw RF recordings, source code, or proprietary detection algorithms require separate justification.
If material records are missing, access expires before review, or events cannot be matched, do not assign an unsupported score. Record the evidence gap and the specific export or explanation needed to resolve it.
7. When Is a Retest Valid, and When Must Comparison Stop?

A retest should address a defined problem, not erase an unfavorable valid result. Agree the retest policy before scoring, including invalid-run criteria, available retest windows, and who authorizes a repeat.
A safety abort or a failed ground-truth recorder may invalidate the affected run under that policy. A detector missing a valid target event is a performance result, not a reason to declare the flight invalid. Record the reason for every exclusion and preserve the original evidence.
When a supplier changes firmware, its protocol library, settings, or installation, identify the revised configuration as a new test baseline. Retest the affected scenarios under the agreed policy. Keep the original and revised results separate; do not silently pool them into a better-looking score.
Apply the same rules and equivalent retest opportunities to all suppliers. Keep a record of the original result, the reason for repetition, the change made, and the new evidence.
If the team cannot reconcile the configuration, scoring rule, or supporting records, mark that comparison “not comparable yet.” That status flags insufficient evidence; it does not mean the product failed.
8. How Should Field-Trial Results Shape the Supplier Shortlist?

Review mandatory capabilities and evidence completeness before comparing performance scores. A high score in an optional feature cannot close an unproven mandatory requirement.
For each supplier, tie the decision to the tested configuration and scenario. Distinguish a demonstrated shortfall from an untested condition or a missing record. Those findings call for different actions: remove a candidate under the agreed criteria, request evidence, run a justified retest, or retain a candidate with a clearly stated condition.
Supplier Comparison and Procurement Decision Record
Copy this record for each supplier, frozen configuration, and requirement. Before scoring, set a pass, hold, or fail rule for every mandatory item. Add a numerical threshold only where the operational requirement calls for one. The row contains instructions, not test results.
| Supplier and configuration/version | Test item | Result and applicable scope | Evidence completeness | Comparability | Open issue | Procurement action |
|---|---|---|---|---|---|---|
| Supplier, device identity, and frozen version | Mandatory/optional item; agreed decision rule and threshold if needed | Observed result, units, target/protocol, scenario, and uncertainty | Run IDs, independent truth, exports, and gaps | Comparable / partially comparable / not yet comparable | Untested condition, deviation, or missing evidence | Shortlist / request evidence / retest / removal; owner and review date |
A shortlist recommendation should state both its supporting evidence and its remaining conditions. Results from the tested equipment at one site do not establish performance for every delivered unit, installation, or target.
For mobile patrols, test whether operators notice alerts as they move between agreed checkpoints and can retrieve event records afterward. Assess RF SKYPOWER’s HZ-HD03 handheld drone detector alongside other portable RF systems for supported target identification, alert presentation, and retrievable records.
Conclusion
A field trial can support a shortlist when suppliers face comparable conditions and each mandatory result can be checked against independent truth. The decision must identify the tested configuration, scoring rule, supporting records, and any unresolved gaps.
This framework cannot supply project-specific performance thresholds or prove suitability for untested targets or sites. The buyer must define performance thresholds from operational needs before scoring. If the scored evidence is complete and comparable, it shows whether each frozen, tested configuration met those thresholds in the evaluated scenarios. A supplier shortlist does not replace acceptance testing of the installed system.
Before inviting suppliers, have engineering, procurement, and safety staff approve the matrix, mandatory decision rules, evidence-access terms, and retest policy. Hold the invitation if a mandatory output still lacks a measurable definition.
Send the trial brief to RF SKYPOWER’s engineering team to review a proposed detector configuration and the evidence it can provide. Include the target/protocol list, required outputs, site and RF conditions, coverage area, handheld or fixed deployment, operator and integration needs, trial timing, scoring criteria, and requested logs or exports. Ask the team to confirm the supported targets, available outputs, and retrievable records before scheduling scored runs.








