4.8.3 smORF evidence

Function

smorf_evidence evaluates Ribo-seq translation evidence for the clustered smORF families produced by smorf_cluster. It is family-aware: for every family and every sample it quantifies P-site abundance, coverage, frame-specific density and frame periodicity, then summarizes the per-family evidence across samples and biological replicate groups.

The final output assigns each family a translation status (Reliable / Uncertain / NoEvidence) and, for reliable families, resolves the translated extent, start site and nested-ORF competition to export a non-redundant reliable smORF table together with its genePred annotation.

Workflow

family index → density cache → sample calibration → chromosome evaluation → merge & competition → annotation export

Stage	Purpose
1	Build family SQLite index
2	Parse density tracks
3	Calibrate samples
4	Evaluate chromosome scaffolds
5	Merge outputs
5b	Resolve translation units
6	Export annotation
7	Report summary

Input files

Input	Required	Description
-i, --family-table	Yes	*.family.message.txt generated by smorf_cluster.
-M, --family-members	Yes	*.family.members.txt generated by smorf_cluster.
--orf-source	Yes	Full smorf_scanner message table that was used by smorf_cluster.
-G, --orf-genepred	Yes	Complete ORF genePred generated by smorf_scanner; column 1 must contain ORF ids matching --orf-source.
-l, --density-list	Yes	Tab-delimited density design file (see below).
-o, --output	Yes	Output prefix.

Density list

The density list is tab-delimited with a header row and the following columns (matched by name, order-independent):

Column	Description
sample	Sample name.
group	Biological replicate group used for replication counting.
strand	+, -, or . (unstranded).
path	Path to the P-site density track (e.g. a bedGraph file).
format	Track format, e.g. bedgraph.

sample  strand  path    format  group
ribo-1  +   ../ribo-1_plus.rpf.bedgraph bedgraph    wt
ribo-1  -   ../ribo-1_minus.rpf.bedgraph    bedgraph    wt
ribo-2  +   ../ribo-2_plus.rpf.bedgraph bedgraph    treat
ribo-2  -   ../ribo-2_minus.rpf.bedgraph    bedgraph    treat

Parameters

Required arguments

Parameter	Default	Description
-i, --family-table		Clustered family table (*.family.message.txt).
-M, --family-members		Family members table (*.family.members.txt).
--orf-source		Full scanner message table used by smorf_cluster.
-G, --orf-genepred		Complete ORF genePred from smorf_scanner.
-l, --density-list		Density design file.
-o, --output		Output prefix.

Evidence arguments

Parameter	Default	Description
--evidence-mode	manual	canonical: calibrate sample-specific thresholds from annotated-ORF controls; manual: use the manual balanced thresholds.
--group	group	Density-list column containing biological replicate groups.
--reliable-sample	2	Minimum number of independent samples supporting a reliable family.

Manual evidence thresholds

Used by --evidence-mode manual:

Parameter	Default	Description
--min-rpf-sum	5.0	Minimum whole-region P-site count.
--min-rpf-per-codon	0.10	Minimum P-sites per codon.
--min-covered-codon	3	Minimum covered codons.
--min-codon-coverage	0.10	Minimum covered-codon fraction.
--moderate-periodicity	0.50	Frame-0 fraction for moderate evidence.
--strong-periodicity	0.60	Frame-0 fraction for strong evidence.
--min-window-rpf	3.0	Minimum sliding-window P-site count.
--min-window-covered-codon	3	Minimum covered codons per window.

Calibration arguments (canonical mode, hidden)

Parameter	Default	Description
--positive-quantile	0.20	Lower-expression quantile of annotated-ORF controls used to derive thresholds.
--positive-min-controls	30	Minimum annotated-ORF controls required; the run fails below this.
--positive-max-controls	5000	Maximum annotated-ORF controls used.

Runtime arguments

Parameter	Default	Description
-t, --thread	1	Requested chromosome workers. The engine may reduce this value (up to 8) and always falls back to a single worker for small inputs.

Output files

With -o gmx4.smorf four files are written:

File	Description
<prefix>.smorf_evidence.txt	Master family-level evidence table (one row per family).
<prefix>.reliable_smorf.txt	Reliable smORF table (one row per reliable ORF).
<prefix>.reliable_smorf.genepred	genePred annotation of the reliable smORFs (ORF ids in column 1).
<prefix>.evidence_summary.txt	metric / value statistics.

Master evidence table (smorf_evidence.txt)

Column groups:

Group	Columns
Family identity	family_id, gene_id, chrom, strand, category, family_type, family_size
Primary-ORF chain	structural_primary, common_body_orf, provisional_primary, quant_primary, evidence_primary
Translation unit	translation_unit_id, translation_unit_index, translation_unit_count, translation_unit_status
Evidence status	evidence_status, family_translation_status, translated_extent_status, extent_supported_sample_count, extent_supporting_samples, extent_supported_bins, extension_rpf_sum, extension_frame0_ratio, family_translation_evidence, reliability_reason
Start site	start_site_status, start_site_reason, start_interval_orf_ids, selection_policy, selection_reason, canonical_anchor_orf, longest_candidate_orf, prior_override_status, leading_support_sample_count, leading_negative_sample_count, noncanonical_extension_support_sample_count, noncanonical_extension_density_ratio
Pooled signal	pooled_extent_rpf_sum, pooled_extent_frame0_ratio, pooled_start_rpf_sum, pooled_end_rpf_sum, pooled_extent_sample_count
Nested competition	nested_competition_status, dominant_parent_orf, competition_overlap_fraction, competition_frame_relation, competition_reason, frame_margin
Replication	reliable_group, supported_sample_count, max_group_sample_support, high_confidence_sample_count, supporting_group_count, supporting_groups, supporting_samples
Best sample	best_sample, best_sample_evidence, best_rpf_sum, best_rpf_per_codon, best_covered_codon, best_coverage_ratio, best_frame0_ratio, best_supported_windows, best_distributed_windows, best_signal_span, best_top_window_fraction, best_localized_only
Members	representative_count, representative_orf_ids, representative_start_codons
Candidate audit	invalid_family_reason, complete_candidate_count, complete_candidate_orf_ids, candidate_audit_status, candidate_failure_reasons
Phase audit	pooled_frame0_density, pooled_frame1_density, pooled_frame2_density, pooled_dominant_frame, suggested_psite_shift_nt, phase_audit_status

Reliable smORF table (reliable_smorf.txt)

A focused view of the master table with one row per reliable ORF. It keeps the primary-chain and evidence columns and adds ORF coordinates: transcript_id, start_codon, stop_codon, nt_length, aa_length, exon_starts, exon_ends.

genePred output

Column 1 is the ORF id; the rest follows genePred conventions with cmpl/cmpl coding status for reliable smORFs.

Evidence classification

Per-sample evidence levels

Level	Description
NoEvidence	No common-body P-site signal in the sample (zero_common_body_rpf).
LowConfidence	Signal present but failing a calibrated threshold: abundance_or_coverage_below_threshold, periodicity_below_calibrated_threshold, localized_only_long_orf_signal, or whole_orf_distribution_not_supported.
MediumConfidence	moderate_common_body_evidence.
HighConfidence	strong_replicable_common_body_evidence.
InvalidFamilyGeometry	The family failed geometry validation and is excluded.

Family translation status

family_translation_status (and evidence_status) is one of:

• Reliable — candidate-first distributed extent evidence, or Medium/High common-body evidence in at least --reliable-sample independent samples.
• Uncertain — signal without sufficient independent-sample replication, localized long-ORF signal, or invalid family geometry.
• NoEvidence — no common-body P-site signal in any sample.

reliability_reason

The reliability_reason column describes how a family was classified:

• common_body_supported_by_N_independent_samples_across_M_groups
• distributed_long_ORF_extent_supported_by_N_independent_samples
• distributed_long_ORF_extent_compatible_by_N_independent_samples
• insufficient_independent_sample_replication
• signal_present_but_no_sample_passed_translation_thresholds
• no_common_body_p_site_signal_in_any_sample
• family_geometry_validation_failed

Reliable smORF definition

A reliable smORF requires a reliable family with a supported or prior-compatible translated extent, a quant_primary, and no dominant ORF within the same transcript translation unit.

Calibration (canonical mode)

• Annotated-ORF families are used as positive controls (between --positive-min-controls and --positive-max-controls).
• For every sample, the lower-expression quantile (--positive-quantile, default 0.20) of the control distribution is used to derive the abundance, coverage and window thresholds.
• Frame-periodicity thresholds are derived from the controls and constrained to [0.40, 0.85] (moderate) and [moderate + 0.05, 0.95] (strong).
• The short/long ORF length split (short_max_codons, long_min_codons) is estimated from the per-category (uORF/dORF/lncORF) length distributions (floored at 20/90 codons), and sliding-window parameters are derived from the controls as well.
• The run fails if fewer than --positive-min-controls annotated-ORF controls are available; keep annotated_ORF in smorf_cluster --keep-categories or switch to --evidence-mode manual.

Examples

cd ./sce/5.smorf/03.evidence

smorf_evidence \
  -i ../02.smorf_cluster/gmx4.reliable.family.message.txt \
  -M ../02.smorf_cluster/gmx4.reliable.family.members.txt \
  --orf-source ../01.smorf_scan/gmx4.message.txt \
  --orf-genepred ../01.smorf_scan/gmx4.genePred \
  --density-list ribo.bedgraph.list \
  --evidence-mode canonical \
  --positive-quantile 0.20 \
  --positive-min-controls 100 \
  --positive-max-controls 5000 \
  --reliable-sample 1 \
  --thread 24 \
  -o gmx4.smorf

Real run on soybean gmx4 (16 samples, 8 replicate groups):

• 1,051,931 families evaluated → 145,167 reliable families (69,641 reliable smORFs), 252,833 uncertain, 653,931 no-evidence.
• 7,146 families failed geometry validation; 37,256 annotated-ORF controls were excluded from the reliable-smORF export.
• Calibration derived window_codons=12, window_step_codons=3, short_max_codons=20, long_min_codons=90.
• The annotated-ORF phase audit suggested a systematic +1 nt P-site shift in 9,766 controls and printed a warning to verify the upstream P-site offset.

Excerpt of the generated evidence_summary.txt:

metric  value
sample_count    16
effective_workers   8
total_families  1051931
reliable_families   145167
uncertain_families  252833
no_evidence_families    653931
reliable_smorfs 69641
invalid_families    7146
reliable_genepred_records   69641
annotated_controls_excluded_from_reliable_smorf 37256
reliable_family_definition  Candidate-first distributed extent evidence or Medium/High common-body evidence in at least 1 independent samples
reliable_smorf_definition   Reliable family with a supported or prior-compatible translated extent, a quant_primary, and no dominant ORF within the same transcript translation unit
uncertain_definition    Signal without sufficient independent-sample replication, localized long-ORF signal, or invalid family geometry
evidence_mode   canonical
group_column    group
reliable_sample 1
...
best_sample_evidence_HighConfidence 72670
best_sample_evidence_MediumConfidence   35395
best_sample_evidence_LowConfidence  224156
best_sample_evidence_NoEvidence 712564
best_sample_evidence_InvalidFamilyGeometry  7146
phase_audit_Frame0Consistent    98108
phase_audit_AlternativeFrameDominant    184252
phase_audit_AmbiguousFrame  98625
phase_audit_NoSignal    663800
annotated_control_suggested_psite_shift_+1_nt   9766
reliability_reason_common_body_supported_by_1_independent_samples_across_1_groups   37009
reliability_reason_insufficient_independent_sample_replication  192176
reliability_reason_no_common_body_p_site_signal_in_any_sample   653931
reliability_reason_signal_present_but_no_sample_passed_translation_thresholds   53511

Notes

• All four upstream inputs are required and must be consistent: the family tables, the scanner message table, and the scanner genePred must originate from the same smorf_scanner / smorf_cluster run.
• In canonical mode keep annotated_ORF in smorf_cluster --keep-categories, otherwise calibration cannot find positive controls.
• Strand-specific P-site bedGraph tracks can be generated by rpf_Bam2bw and listed one row per sample per strand.
• The annotated-ORF phase audit compares the dominant frame of controls against frame 0. A systematic alternative frame (many controls sharing the same suggested P-site shift) triggers a warning; verify the upstream P-site offset before interpreting candidate reading frames.
• The engine caps chromosome workers at 8 regardless of --thread.
• The master evidence table is large (one row per family); use reliable_smorf.txt for downstream prioritization.