Implicit Neural Representations (INRs) have shown great potential for representing complex signals as continuous functions, offering parameter efficiency. However, training INRs for even a single signal can be computationally expensive, hindering scalability for large datasets. Optimization-based meta-learning has emerged as a promising approach to accelerate INR training, rapidly improving signal reconstruction within a few optimization steps. Unfortunately, these methods suffer from poor scalability with signal dimensionality, as the required context set (coordinate and signal value pairs) grows super-linearly. This memory bottleneck limits the number of gradient steps and restricts the application to high-dimensional signals. Existing attempts to address this, such as dividing signals into patches, increase adaptation time and ignore cross-patch statistics. This paper proposes a solution by exploring the possibility of reducing the context set size without compromising adaptation performance, drawing inspiration from data pruning techniques.

The paper reviews existing work on INRs, highlighting their advantages (parameter efficiency, suitability for various modalities) and the challenges in scaling them to high-dimensional data. It discusses prior meta-learning techniques for INRs, such as Learnit and TransINR, emphasizing their memory limitations. The authors also review related work in efficient meta-learning (first-order MAML, Reptile, continual trajectory shifting), and sparse data selection techniques (data pruning, memory-based continual learning, active learning), setting the stage for their proposed approach.

ECoP introduces a three-pronged approach to efficient meta-learning for INRs: 1. **Error-based Online Context Pruning:** At each inner loop iteration, ECoP adaptively selects a subset of high-error coordinate-value pairs from the full context set. This selection is guided by an error metric (similar to the EL2N score), focusing on high-loss elements. This strategy prioritizes global structure initially and refines high-frequency details later. The sampling ratio (γ) controls the trade-off between performance and memory efficiency. 2. **Bootstrapped Correction:** To compensate for potential information loss from context pruning, ECoP employs a bootstrapped target model. After adapting the INR meta-learner for K steps using the pruned context set, it further adapts the model for L steps using the full context set, creating a bootstrapped target (θbootK+L). The meta-learner is then regularized to minimize the parameter distance between the pruned-context model (θK) and the bootstrapped target. This correction is computationally efficient as it avoids saving intermediate gradients. 3. **Test-time Gradient Scaling:** Since the gradient norms differ between meta-training (using pruned context) and meta-testing (using the full context), ECoP scales the test-time gradient using the ratio of gradient norms from the pruned and full context sets. This ensures consistent gradient updates and improves performance. The overall meta-learning objective combines the reconstruction error on the full context set and the parameter distance to the bootstrapped target. The algorithm uses a first-order adaptation method during testing with the full context set. The algorithm is model agnostic, meaning it adapts to various architectures.

The experimental results demonstrate ECoP's effectiveness across various modalities (image, video, audio, manifold). ECoP consistently outperforms baseline methods (Learnit, TransINR, FOMAML, Reptile, random initialization) in terms of reconstruction performance (measured by PSNR, SSIM, LPIPS). The improvements are particularly significant for high-resolution signals where existing methods often fail due to memory constraints. ECoP shows superior performance even on high-resolution videos (256x256x32) and images (1024x1024), which are computationally expensive for baseline methods. Ablation studies confirm the contribution of each component of ECoP (context pruning, bootstrapped correction, gradient scaling). ECoP enables longer adaptation horizons, avoiding the myopia of short-horizon meta-learning, which is demonstrated through both quantitative and qualitative analysis. The analysis of the loss statistics of coordinates supports the effectiveness of the error-based pruning scheme. Finally, the paper shows that ECoP is more efficient in terms of training time compared to Learnit when achieving the same performance level. Cross-domain adaptation experiments demonstrate that the meta-learned initialization by ECoP is transferable across various datasets and modalities.

The results confirm that ECoP effectively addresses the memory limitations of optimization-based meta-learning for INRs while maintaining or even improving reconstruction performance. The success of ECoP highlights the importance of efficient context selection and information preservation techniques in meta-learning for high-dimensional data. The model-agnostic nature of ECoP makes it broadly applicable to different INR architectures and data types. The superior performance on high-resolution data opens doors for new applications that were previously infeasible due to memory constraints. The transferability of learned initializations across domains indicates ECoP learns meaningful representations.

ECoP is a significant advancement in efficient meta-learning for INRs. Its innovative use of error-based context pruning, bootstrapped correction, and gradient scaling leads to substantial improvements in memory efficiency and reconstruction performance across various modalities and resolutions. Future work could explore extending ECoP to scenarios with disjoint context and target sets (e.g., scene rendering) and further scaling it to extreme high-resolution signals (e.g., long 8K videos) using techniques like iterative tree search.

While ECoP demonstrates significant improvements, several limitations exist. The hyperparameters (γ, λ, L) need careful tuning, although the authors mention they observed relative insensitivity across diverse datasets and architectures. The study primarily focuses on scenarios where the inner and outer loop optimization use the same context set; extending it to cases with disjoint sets warrants further investigation. The experiments were performed on specific hardware; scaling to diverse hardware requires further analysis.