RF drone detection system on a tripod during a supplier field trial, with an observer, laptop operator, and drone overhead.

A supplier field trial for an RF drone detection system needs comparable target opportunities, independent ground truth, and records buyers can audit. Use it to shortlist credible candidates before award; acceptance of the installed system comes later.

A polished demonstration may show a convincing alert without proving what caused it, whether the aircraft was identified correctly, or how often the detector missed a target. Define the outputs the operation needs first. Then hold the test conditions and scoring rules constant across suppliers.

1. What Must an RF Drone Detection Supplier Field Trial Prove?

Passive RF drone detector receiving signals from an aircraft and controller, contrasted with an RF-silent drone.

Turn the operational requirement into specific outputs before discussing detection range. An alert, an identity, a position, and a track answer different questions.

Score detections, classifications, identifications, positions, and tracks as distinct outputs. Classification may assign a category, while identification may name a model, protocol, or individual identifier. An alert does not establish identity, and a single position is not a continuing track. Location records must distinguish the aircraft, its controller, and a reported takeoff point; none automatically establishes a person’s current position.

Passive RF detection needs a relevant signal from the aircraft or associated equipment that the detector can receive. Record RF-silent and unsupported autonomous targets as explicit coverage cases. If neither source emits such a signal, a passive RF-only system cannot detect that target. Where this coverage is mandatory, evaluate complementary sensing or record the RF-only capability gap; do not quietly exclude the target from the procurement decision.

The procurement team should separate mandatory capabilities from useful extras. For each required output, name the target categories, operating area, response task, and available staff. The engineering team can then define a measurable objective and the evidence needed to judge it. “Provide an alert during the agreed target exposure window” is testable; “detect drones reliably” is not yet a scoring rule.

Before inviting suppliers, decide whether RF, radar, or multi-sensor detection fits the mandatory target list. That decision determines which equipment belongs in the field trial.

2. How Do You Build a Common Test Matrix for All Suppliers?

Overhead field trial layout showing a common drone route, two supplier receiver positions, and recorded antenna setups.

Set the same operational tasks and scoring definitions for every supplier. Allow different approved mounts where necessary, but record antenna position, height, orientation, configuration, and site conditions that could affect results.

Give suppliers a commissioning period, then freeze each tested configuration. Before scoring, fix target-opportunity counts for each required target, protocol, and scenario, plus no-target monitoring hours under defined RF conditions. If the team uses sequential testing, approve a statistically valid uncertainty method and stopping rule in advance. Group related passes by flight, time block, and RF condition; repeated passes are not automatically independent samples. Balance supplier order and time slots. Never add scored runs selectively after reviewing early results.

This matrix is a planning template. Fill in routes, distances, heights, repetitions, and scoring windows for the agreed targets and protocols before issuing the trial schedule.

Common Supplier Trial Matrix

Scenario / run identifierTarget and protocolDeployment and flight conditionsRequired outputRepetition and scoring rulesGround-truth and log sources
Baseline approach; one ID per repeatNamed drone, controller, protocol, and modeRoute, altitude, direction, coverage zone, and antenna setupRequired alert, identity, position, or trackFrozen repeats and event-matching rulesIndependent flight/observer truth and detector export
Changed approach or obstruction; separate run IDsSame agreed target/protocol combinationsDirection, terrain, or line-of-sight change; other conditions loggedSame required outputs under changed geometryScore separately from baselinePosition truth, obstruction notes, and system logs
Busy RF environment; separate run IDsNamed target and operating modeRecorded ambient RF activity and installation setupRequired outputs amid background signalsRepeats under documented RF conditions; no unqualified poolingTarget truth, RF-background record, and event logs
Concurrent targets, when required; separate run IDsNamed target group, protocols, and modesSimultaneous activity, target separation, and RF backgroundPer-target alerts, identities, or tracksCount misses, merged targets, and identity switchesIndependent time-aligned truth for each target and system logs
No-target monitoring; one ID per windowNo planned target; log other observed activityAgreed area, monitoring hours, and RF backgroundAll alerts for later adjudicationCount alarm episodes over valid online time; retain unresolved eventsIndependent observation, availability, and alert logs
Controller powered before takeoff; separate event IDsNamed drone, controller state, and protocolPower-on, ground activity, and takeoff recorded separatelyRF-presence output versus airborne-target claimApply the agreed output rule; do not assume an RF alert is falseController/activity truth and timestamped alerts

Record unsupported target categories before testing. If one is mandatory, keep that gap in the decision record even when the supplier performs well on supported targets.

Approve valid-run rules, flight safety, site permissions, and lawful RF and data collection before testing. For US trials, seek qualified legal review of RF monitoring, decoding, and data retention. At airports, check FAA coordination requirements before deployment. Check the detector’s actual receive and transmit functions, including setup, update, or calibration modes; a passive label alone does not establish that every mode is non-transmitting. This is a detection trial and does not require interference or mitigation exercises.

3. How Do You Establish Independent Ground Truth?

Timeline showing controller activation, an RF alert before drone takeoff, and clock checks around the scored trial.

Ground truth is the independent reference for what actually happened, when it happened, and where. It must not come solely from the detector being evaluated.

Use authorized aircraft flight records, suitable independent position references, and trained observers as the task requires. Record the reference method’s uncertainty and any gaps. An observer may confirm takeoff and landing without providing the position accuracy needed to assess a supplier’s location claim.

Check clock alignment before and after each scored test block across the truth source, detector, recording system, and observers. Record any offset or drift that could affect event matching or latency. Set the time basis and allowed matching window. For spatial matching, specify whether a target opportunity starts when the aircraft enters the agreed coverage zone.

Match each scored system output to the correct target and event. The matching rule should resolve duplicate alerts and ambiguous associations without relying on the vendor operator’s recollection.

Consider a controller that starts transmitting before takeoff. An early RF alert may correctly indicate radio activity. It does not, by itself, show that an aircraft was airborne. The truth record must distinguish those events before the evaluator scores the output.

If clock uncertainty, missing truth, or position-reference error prevents a defensible judgment, pause scoring for the affected metric. A supplier’s own map marker cannot independently validate its location accuracy.

4. How Should Operator Roles and Blind Trials Be Controlled?

Drone detection field trial with safety staff, an independent recorder, and vendor operators in a separate blind test station.

Assign responsibility before the trial begins. The trial director controls the approved plan and records deviations. The pilot and safety team control flight safety. An independent recorder maintains ground truth, the vendor operator runs the detector, and the scoring/data custodian preserves records and applies the agreed rules.

If one person fills several roles, document the overlap and independent checks. The vendor must not be the sole recorder of target activity or the judge of its own alerts.

Before scoring, record each device identity, software and firmware versions, protocol-library version if applicable, and active configuration.

For blind scored runs, withhold exact event timing and sequence from the vendor operator where practical. Keep the pilot and safety team fully informed of the approved flight plan. Blinding must never remove safety information or prevent an abort.

Record who operated the system, what assistance they received, and every adjustment during scored activity. Separate supplier-assisted operation from buyer-operated trials if both matter to procurement. Without that distinction, supplier coaching may inflate the apparent system performance.

Apply the agreed deviation rule to any scored run compromised by a cue, intervention, or configuration change. Keep that run separate from unchanged blind trials.

5. Which RF Detection Metrics Can Actually Be Compared?

Scoring timeline showing five eligible drone detection opportunities, four matched alerts, one outage miss, and an unresolved alert.

Compare metrics only when their definitions, eligible events, target conditions, and evidence align. An attractive number with a different denominator answers a different question.

Agree the definitions below before suppliers see the scored results.

Performance Metric Definitions and Comparability

MetricEligible events and calculationTime, space, and target scopeRequired evidenceNot directly comparable when
End-to-end detection success rateMatched detections / all eligible opportunities, including detector outagesTarget/protocol, scenario, zone, and windowIndependent truth, matching rules, detection and availability logsTarget sets, misses, or exposure windows differ
Detection rate while operationalMatched detections / eligible opportunities while all required functions are readySame target conditions and frozen definition of readyTruth, detection events, and availability timelineOperating state or eligible opportunities differ
Detector availabilityTime all required detection and alert functions are ready / required monitoring time; pre-agreed exclusions onlyAgreed duty periods and ready-state definitionHealth logs and independent monitoring timelineDevice-on is treated as ready, or outages are hidden
Confirmed false-alarm rate; unresolved alertsConfirmed false-alarm episodes / valid online hours; unresolved events and alarm duration separatelyScored output type, observed area, and RF conditionsAlert records, independent observations, and availability logsAlert grouping, truth coverage, exclusions, or unknowns differ
Detection latencyFirst qualifying output time minus agreed reference-event time; misses separatelySame start event, output stage, and target scenarioAligned truth/output timestamps and clock-offset recordStart events differ or pre-existing alerts are mishandled
Identification correctness and availabilityFreeze one ID decision per eligible opportunity; correctness = correct issued decisions / all scored issued decisions; availability = opportunities with an in-window ID at the required level / all eligible identity opportunities. Report wrong, conflicting, missing, and corrected outcomes separatelySame identity level, target/protocol set, and counting unitKnown identity and complete identity-output historyCorrected IDs hide earlier errors, or ID levels differ
Location error and availabilityError = agreed 2D/3D distance (m) between each valid system position and time-matched independent truth, with distribution summaries and sample counts; availability = eligible target time with valid, sufficiently recent positions / all eligible target time. Report missing and stale positions separatelySame located object, coordinate basis, 2D/3D measure, and output-age limitIndependent position truth and uncertainty; system position historySummary statistics differ, or aircraft and controller locations are mixed
Track continuityValid associated-track time / eligible target time; also gaps and identity switchesSame association, update, and gap rulesTime-aligned target truth and track historyGap or target-association rules differ

Detection Probability and False Alarm Measurement

Define one detection opportunity before counting results. It may be a passage through a specified zone. Several screen updates during that passage still represent one opportunity.

Report each target, protocol, and scenario before any aggregate, along with counts and the agreed uncertainty method. Nine detections in ten independent opportunities and ninety in one hundred both yield 90%; the larger independent sample supports a tighter uncertainty range. Repeated flights in the same conditions may not provide that independence. Neither result proves performance on untested targets.

The end-to-end rate includes eligible opportunities missed during outages. Report it with detection rate while operational and detector availability. A detector that performs well only when online may still leave unacceptable monitoring gaps.

Define false alarms against the claimed output. An RF-presence indication, airborne-drone alert, and model identification have different truth conditions. Freeze the episode-grouping rule and record both count and duration. Valid monitoring hours require known detector availability and enough independent observation to judge alerts. A window with no planned target may still contain other aircraft or RF activity.

Keep alerts without adequate truth as unresolved. Report confirmed false-alarm episodes per valid online hour, unresolved counts, total alert burden, and hours excluded for truth gaps or downtime. A confirmed-only rate understates the possible burden when unresolved alerts may be false. Hold the comparison if those unknowns could change the ranking.

Zero confirmed false alarms during a short observation does not establish a zero long-term false-alarm rate. Report valid observed hours and downtime; never use denominator exclusions to hide system unavailability.

Detection Latency, Identity, and Location Accuracy

Choose the latency endpoint the operation needs: first system event, operator-visible alert, or another specified output. Handle alerts already active at the reference event under a frozen rule; they must not create misleading zero or negative latency. Show misses beside latency for successful detections.

Keep wrong and conflicting identity labels, even when the system corrects them later. For location, agree on a summary such as the median and a high percentile in advance; also report sample counts, stale outputs, and reference uncertainty. A correct model label is not a unique aircraft ID; accurate positions from a few moments do not show continuous availability.

Compare metrics only where their definitions and conditions align. If only some evidence qualifies, compare that portion and state the remaining gap. Hold the affected ranking when event definitions or reference data remain unresolved.

6. What Logs and Data Rights Must Buyers Secure?

Evidence workflow linking system configuration, independent ground truth, and detector exports by run ID while preserving original files.

Agree the evidence package and buyer access rights before the trial. A dashboard demonstration is not enough if the evaluation team cannot later reconstruct what happened.

Each run needs a traceable connection between the target, truth record, and tested configuration. Request device identities, software/firmware and relevant protocol-library versions, active settings, run IDs, timestamps, alerts, identities, positions or tracks, errors, and operator changes. Record which requested outputs the system does not provide.

Check a sample export before scoring. The evaluation team should be able to interpret the format, field definitions, time basis, coordinate system, and missing-value codes. It should also be able to match exported events to the independent truth record.

Define who owns each dataset and what the buyer may retain, analyze, and share with an authorized evaluator. Agree export delivery, retention periods, access after the trial, and confidentiality controls for sensitive location and operator information. Keep original exports intact and document any later corrections or processing.

Request enough evidence to audit the result. Raw RF recordings, source code, or proprietary detection algorithms require separate justification.

If material records are missing, access expires before review, or events cannot be matched, do not assign an unsupported score. Record the evidence gap and the specific export or explanation needed to resolve it.

7. When Is a Retest Valid, and When Must Comparison Stop?

Decision flow retaining valid misses, assessing ground-truth failures, and separating original results from revised configuration retests.

A retest should address a defined problem, not erase an unfavorable valid result. Agree the retest policy before scoring, including invalid-run criteria, available retest windows, and who authorizes a repeat.

A safety abort or a failed ground-truth recorder may invalidate the affected run under that policy. A detector missing a valid target event is a performance result, not a reason to declare the flight invalid. Record the reason for every exclusion and preserve the original evidence.

When a supplier changes firmware, its protocol library, settings, or installation, identify the revised configuration as a new test baseline. Retest the affected scenarios under the agreed policy. Keep the original and revised results separate; do not silently pool them into a better-looking score.

Apply the same rules and equivalent retest opportunities to all suppliers. Keep a record of the original result, the reason for repetition, the change made, and the new evidence.

If the team cannot reconcile the configuration, scoring rule, or supporting records, mark that comparison “not comparable yet.” That status flags insufficient evidence; it does not mean the product failed.

8. How Should Field-Trial Results Shape the Supplier Shortlist?

Procurement decision chart using mandatory requirements and evidence completeness to shortlist, request evidence, retest, or remove suppliers.

Review mandatory capabilities and evidence completeness before comparing performance scores. A high score in an optional feature cannot close an unproven mandatory requirement.

For each supplier, tie the decision to the tested configuration and scenario. Distinguish a demonstrated shortfall from an untested condition or a missing record. Those findings call for different actions: remove a candidate under the agreed criteria, request evidence, run a justified retest, or retain a candidate with a clearly stated condition.

Supplier Comparison and Procurement Decision Record

Copy this record for each supplier, frozen configuration, and requirement. Before scoring, set a pass, hold, or fail rule for every mandatory item. Add a numerical threshold only where the operational requirement calls for one. The row contains instructions, not test results.

Supplier and configuration/versionTest itemResult and applicable scopeEvidence completenessComparabilityOpen issueProcurement action
Supplier, device identity, and frozen versionMandatory/optional item; agreed decision rule and threshold if neededObserved result, units, target/protocol, scenario, and uncertaintyRun IDs, independent truth, exports, and gapsComparable / partially comparable / not yet comparableUntested condition, deviation, or missing evidenceShortlist / request evidence / retest / removal; owner and review date

A shortlist recommendation should state both its supporting evidence and its remaining conditions. Results from the tested equipment at one site do not establish performance for every delivered unit, installation, or target.

For mobile patrols, test whether operators notice alerts as they move between agreed checkpoints and can retrieve event records afterward. Assess RF SKYPOWER’s HZ-HD03 handheld drone detector alongside other portable RF systems for supported target identification, alert presentation, and retrievable records.

Conclusion

A field trial can support a shortlist when suppliers face comparable conditions and each mandatory result can be checked against independent truth. The decision must identify the tested configuration, scoring rule, supporting records, and any unresolved gaps.

This framework cannot supply project-specific performance thresholds or prove suitability for untested targets or sites. The buyer must define performance thresholds from operational needs before scoring. If the scored evidence is complete and comparable, it shows whether each frozen, tested configuration met those thresholds in the evaluated scenarios. A supplier shortlist does not replace acceptance testing of the installed system.

Before inviting suppliers, have engineering, procurement, and safety staff approve the matrix, mandatory decision rules, evidence-access terms, and retest policy. Hold the invitation if a mandatory output still lacks a measurable definition.

Send the trial brief to RF SKYPOWER’s engineering team to review a proposed detector configuration and the evidence it can provide. Include the target/protocol list, required outputs, site and RF conditions, coverage area, handheld or fixed deployment, operator and integration needs, trial timing, scoring criteria, and requested logs or exports. Ask the team to confirm the supported targets, available outputs, and retrievable records before scheduling scored runs.