Journal of Computing, Intelligence and Information Sciences
The Journal of Computing, Intelligence and Information Sciences (JCIIS) is a peer-reviewed scholarly journal dedicated to advancing research and innovation across computing, artificial intelligence, information sciences, data-driven technologies, and emerging digital systems. The journal provides an interdisciplinary platform for researchers, academics, practitioners, and industry professionals to disseminate original research, systematic reviews, methodological advances, and applied studies addressing contemporary challenges in computing and intelligent information systems.
Hardware-Aware Efficient Inference for Long-Context and Reasoning Language Models: Low-Bit Quantization, KV-Cache Compression, and Speculative Decoding
The deployment cost of large language models increasingly depends not only on parameter count but also on context length, reasoning-token generation, key-value cache growth, accelerator memory bandwidth, batch size, and decoding strategy. These pressures are amplified in reasoning models, which may generate thousands of intermediate tokens, and in long-context systems, where persistent key-value states can dominate runtime memory consumption. This review examines hardware-aware language-model inference techniques through September 2026. Post-training quantization methods including GPTQ, SmoothQuant, AWQ, and low-bit weight formats substantially reduced parameter-memory requirements. BitNet investigated extremely low-bit model representations trained natively rather than quantized after training. FlashAttention and FlashAttention-2 improved exact attention by restructuring memory access, while vLLM's PagedAttention improved key-value-cache management for concurrent serving. StreamingLLM and related long-context methods highlighted the importance of cache organization for persistent generation. KIVI and subsequent KV-cache quantization methods targeted the rapidly growing inference-state memory associated with long contexts. Speculative decoding reduced latency by allowing smaller draft models to propose tokens verified by a larger target model, while Medusa and EAGLE expanded multi-token prediction and draft-generation strategies. By 2026, hardware-aware inference increasingly involved adaptive precision and sensitivity-aware cache compression. ChanMix introduced mixed-precision allocation across key-value-cache channels for long-context inference. AnyBCQ proposed flexible hardware-oriented binary-coded quantization enabling runtime multi-precision operation. Hierarchical Block Quantization explored block-level hardware efficiency alongside second-level scaling, while asynchronous test-time scaling demonstrated that reasoning throughput can also be improved through speculative and asynchronous execution. The review argues that model quality and inference efficiency must increasingly be co-designed across algorithms, numeric formats, kernels, cache architectures, and serving systems.
Copyright (c) 2026 Journal of Computing, Intelligence and Information Sciences
World Models and Vision-Language-Action Systems for Robotic Intelligence: Predictive Simulation, Policy Evaluation, and Whole-Body Control
World models seek to represent how an environment evolves and thereby allow an intelligent system to predict the consequences of possible actions before executing them. Although latent world models have long been studied in reinforcement learning, advances in generative video, multimodal perception, and vision-language-action modeling have expanded the concept toward realistic physical simulation and robot control. This review examines world models and vision-language-action systems for robotics through September 2026. Early latent-dynamics methods demonstrated that agents could learn policies using imagined trajectories, while Dreamer-style systems progressively improved sample-efficient control through learned representations of environmental dynamics. Diffusion Policy introduced expressive diffusion-based action generation for visuomotor behavior. RT-1, RT-2, Open X-Embodiment, and Octo demonstrated increasingly general robotic policies trained across heterogeneous datasets and tasks. Generative video systems subsequently offered richer visual predictions of object and environment dynamics. By 2025–2026, this line of work increasingly converged with world modeling. PhysWorld explored robot learning from physical video-generation models. WorldGym used an action-conditioned video model as an environment in which robotic policies could be evaluated through simulated rollouts and vision-language-based rewards. Sequential World Models extended model-based reinforcement learning to multi-robot cooperation through autoregressive agent-wise dynamics. WholeBodyVLA integrated vision-language-action learning with latent actions and reinforcement-learning control for humanoid loco-manipulation. Research on video generation for robotics and world-action models increasingly formalized the transition from predicting future observations toward using predicted futures to determine executable actions. The review identifies physical fidelity, causal correctness, action grounding, sim-to-real transfer, long-horizon error accumulation, and safety as central challenges.
Multimodal Computer-Use Agents for Web and Desktop Environments: Cognitive Reasoning, Verification, Robustness, and Interactive Evaluation
Language-model agents increasingly operate through graphical interfaces rather than interacting only with text-based APIs. Computer-use agents must perceive screenshots, interpret textual and visual interface elements, reason over user goals, select actions, recover from interface changes, and determine whether an action has successfully achieved the intended outcome. This review examines multimodal computer-use agents and their evaluation through September 2026. Earlier web-navigation benchmarks such as WebShop, Mind2Web, WebArena, and WorkArena progressively increased environmental realism, while OSWorld extended interaction to complete operating-system environments. AgentBench provided broader interactive evaluation across several agent environments. Advances in multimodal large language models enabled agents to rely increasingly on pixels rather than structured accessibility trees or pre-extracted HTML. By 2025–2026, research shifted toward cognitive structure, robustness, verification, and safety. Web-CogReasoner introduced a framework separating factual, conceptual, and procedural knowledge for multimodal web-agent reasoning. ScienceBoard evaluated computer-using agents in realistic scientific workflows. EgoBench introduced interactive egocentric multimodal tasks requiring perception, tool invocation, multi-hop reasoning, and user interaction. AgentNoiseBench examined robustness under noisy user and tool conditions. Research on multimodal verifiers explored whether vision-language models could reliably judge outcomes across computer-use and robotic trajectories. BLIND-ACT demonstrated a tendency toward blind goal-directedness, in which computer-use agents continue pursuing goals despite ambiguity, infeasibility, or contextual reasons to stop. The review compares visual grounding, cognitive planning, action representation, tool invocation, multimodal verification, robustness testing, and safety. Reliable computer use requires not only stronger perception and reasoning but also explicit mechanisms for uncertainty, refusal, progress checking, and recovery.
Persistent Memory and Experience Learning for Autonomous Language Agents: Retrieval, Consolidation, Reflection, and Selective Forgetting
Long-lived autonomous agents require mechanisms for retaining and reusing information that extends beyond a single model context window. While conventional retrieval-augmented generation primarily supplies external factual documents to a language model, agent memory introduces additional requirements involving experience accumulation, temporal ordering, personalization, reflection, contradiction handling, selective retention, and learning from past actions. This review examines persistent memory for language-model agents through September 2026. Early retrieval-based architectures, episodic simulations, reflective agents, and external-memory systems established mechanisms for storing interaction histories and reintroducing them into later reasoning. Generative Agents demonstrated the use of memory streams, retrieval, reflection, and planning in persistent simulated environments. Reflexion used linguistic self-feedback as episodic memory for iterative task improvement, while Voyager accumulated reusable skills during open-ended embodied interaction. MemGPT conceptualized external memory management through hierarchical movement between limited context and persistent storage. By 2025–2026, research increasingly shifted from demonstrating that memory could help agents toward evaluating what effective memory actually requires. MemoryAgentBench emphasized accurate retrieval, test-time learning, long-range understanding, and selective forgetting. AMemGym introduced interactive on-policy evaluation for memory-driven assistants. Retrieval-Augmented LLM Agents combined training with retrieved trajectories to improve transfer to unseen tasks. Agent Learning via Early Experience investigated learning from environment transitions and self-reflection without relying entirely on conventional reinforcement learning. DreamGym scaled agent learning using synthesized experience and simulated environment dynamics. The review analyzes memory writing, retrieval policy, reflection, consolidation, forgetting, provenance, personalization, and privacy. Persistent memory is emerging as a core architectural layer separating short-lived language interaction from adaptive autonomous behavior.
Adaptive Test-Time Reasoning in Large Language Models: Dynamic Compute Allocation, Overthinking Control, Verification, and Efficient Deliberation
The development of reasoning-oriented large language models shifted the scaling problem from exclusively increasing training-time model capacity toward determining how much computation should be allocated to individual problems during inference. Early test-time scaling methods generally assumed that additional reasoning tokens, sampled solution paths, or verification steps would improve difficult-task performance. Evidence available by 2026, however, increasingly demonstrated that indiscriminate reasoning expansion can produce substantial inefficiency and may even reduce accuracy through overthinking, redundant exploration, or poor allocation of reasoning effort. This review examines adaptive test-time reasoning methods through September 2026. Chain-of-thought prompting, self-consistency, verifier-guided reasoning, Tree-of-Thoughts, iterative refinement, and process supervision are reviewed as foundations for inference-time deliberation. DeepSeek-R1 and related reinforcement-learned reasoning systems demonstrated that extended reasoning behavior could emerge through optimization targeted toward verifiable tasks. Subsequent work investigated compute-optimal scaling and controllable reasoning budgets. Plan-and-Budget introduced problem decomposition and adaptive token allocation to reduce both overthinking and underthinking, while OptimalThinkingBench explicitly evaluated whether models select an appropriate amount of reasoning for tasks of different difficulty. Research on training-data requirements further indicated that additional inference computation is useful only when prerequisite capabilities exist in the learned model. Asynchronous Test-Time Scaling examined latency-efficient parallel and sequential reasoning using speculative mechanisms and conformal prediction. The review concludes that the central challenge is no longer simply inducing longer reasoning traces, but learning or estimating when reasoning is required, how reasoning effort should be distributed, when verification is useful, and when computation should terminate.
World Foundation Models and Large-Scale Generative Video: From Diffusion-Based Synthesis to Interactive Physical-AI Simulation
Generative video modeling evolved rapidly from short text-conditioned video synthesis toward increasingly general models capable of representing temporal dynamics, physical interactions, camera motion, and interactive environments. This progression renewed interest in world models: learned representations that predict future states of an environment and can potentially support planning, robotics, simulation, and embodied artificial intelligence. This review examines the convergence of generative video and world modeling through 2025. Early world-model research demonstrated that compact learned dynamics could support control within imagined environments, while Dreamer-style systems learned behaviors directly through latent dynamics. The development of diffusion models and latent video representations subsequently improved high-fidelity visual synthesis. Make-A-Video transferred text-to-image knowledge into video generation without paired text-video data, while Stable Video Diffusion emphasized scalable dataset curation and latent video diffusion. Lumiere introduced a space-time architecture for coherent full-video generation, and VideoPoet treated video synthesis as multimodal autoregressive generation. Genie advanced the concept of generative interactive environments by learning latent actions from unlabeled video. CogVideoX and HunyuanVideo scaled diffusion-transformer architectures for longer and higher-quality videos. In 2025, Wan provided an open family of large-scale video generation models spanning multiple generation and editing tasks. The Cosmos World Foundation Model platform explicitly positioned generative world models as infrastructure for physical AI, combining video tokenization, pretrained world models, and post-training workflows. WorldSimBench subsequently proposed perceptual and embodied evaluation of video generation systems as world simulators. The review examines temporal coherence, action conditioning, visual tokenization, diffusion Transformers, latent dynamics, physical consistency, evaluation, and simulation-to-action transfer. Major limitations include incorrect physical dynamics, long-horizon drift, expensive data requirements, temporal hallucination, safety, and weak causal understanding despite photorealistic outputs.
Copyright (c) 2025 Journal of Computing, Intelligence and Information Sciences
Multimodal Reasoning and Visual Agents: Native Vision-Language Pretraining, Document Intelligence, Long-Video Understanding, and Computer Interaction
Multimodal large language models evolved rapidly from systems that primarily described individual images into increasingly general visual reasoning models capable of processing documents, diagrams, charts, long videos, multiple images, spatial information, and graphical user interfaces. This review examines the progression toward multimodal reasoning and visual agents through 2025. CLIP established scalable image-text alignment, while Flamingo demonstrated few-shot multimodal learning through the integration of pretrained visual and language components. BLIP-2 reduced multimodal training requirements by introducing a compact bridge between frozen image encoders and language models. LLaVA and InstructBLIP subsequently demonstrated the importance of multimodal instruction tuning. Models such as Qwen-VL and InternVL extended capabilities toward localization, multilingual understanding, and higher-resolution visual processing. By 2024, Qwen2-VL, LLaVA-OneVision, PaliGemma, and Molmo expanded dynamic-resolution, multi-image, and video capabilities. The 2025 generation further emphasized native multimodal pretraining and interaction with external environments. Qwen2.5-VL combined dynamic visual resolution, document parsing, object localization, temporal encoding, long-video comprehension, and visual-agent capabilities. InternVL3 introduced native multimodal pretraining rather than adapting only a pretrained text model, while Phi-4-Multimodal used modality-specific adapters to integrate vision and speech with a compact language backbone. Gemma 3 extended lightweight open models with image understanding, multilinguality, and long context. This review compares visual tokenization, dynamic resolution, connector architectures, multimodal pretraining, instruction tuning, temporal encoding, document reasoning, grounding, and interface interaction. Key limitations include hallucinated visual details, weak fine-grained spatial reasoning, OCR errors, temporal inconsistency, interface fragility, and limited evaluation of real-world agentic reliability.
Autonomous Software Engineering Agents for Repository-Level Code Maintenance, Debugging, and Issue Resolution
Large language models initially transformed software development through code completion and natural-language-to-code generation, but by 2025 research increasingly focused on autonomous software-engineering systems capable of navigating complete repositories, interpreting issue reports, executing commands, modifying multiple files, running tests, and iteratively correcting failures. This review examines the progression from pretrained code models to repository-level software-engineering agents through 2025. CodeBERT, GraphCodeBERT, CodeT5, and UniXcoder established transferable representations for source-code understanding and generation, while Codex and AlphaCode demonstrated substantial improvements in program synthesis. Repository-level benchmarks subsequently exposed limitations that were obscured by isolated programming tasks. RepoBench evaluated cross-file retrieval and completion, while SWE-bench formulated real GitHub issue resolution as an executable benchmark. SWE-agent showed that carefully designed agent-computer interfaces could substantially affect autonomous repair performance, whereas AutoCodeRover combined large language models with structured code-search mechanisms. Agentless demonstrated that comparatively simple localization, repair, and validation pipelines could remain competitive with more complex autonomous systems. OpenHands provided an open platform in which agents could write code, operate terminals, and browse the web. In 2025, SWE-Lancer expanded evaluation to real freelance engineering tasks with monetary value, while SWE-smith addressed the scarcity of scalable training environments for software-engineering agents. The review compares issue localization, repository retrieval, planning, patch generation, execution feedback, test-driven validation, sandboxing, cost, and benchmark design. Major challenges include long-horizon failure, incomplete specifications, flaky tests, unsafe command execution, dependency management, environment reproduction, benchmark contamination, and the gap between benchmark resolution rates and dependable professional software engineering.
Test-Time Scaling and Reinforcement-Learned Reasoning in Large Language Models: Verifiers, Budget Forcing, and Emergent Deliberation
The rapid development of reasoning-oriented large language models during 2024 and 2025 shifted attention from scaling only pretraining parameters toward allocating additional computation during inference. Test-time scaling allows a model to generate, evaluate, revise, or search among multiple reasoning trajectories before committing to a final answer. This review examines the foundations and principal developments in inference-time reasoning through 2025. Chain-of-thought prompting established that intermediate natural-language reasoning could substantially improve performance on complex tasks, while self-consistency demonstrated the value of sampling multiple reasoning trajectories and aggregating their answers. Self-Taught Reasoner and Quiet-STaR explored mechanisms through which models could learn to generate useful internal rationales. Process-supervision and verifier research further demonstrated the importance of evaluating intermediate reasoning rather than relying solely on final-answer correctness. Tree-of-Thoughts and iterative self-refinement extended reasoning toward explicit search and revision. Research on compute-optimal test-time scaling subsequently formalized the relationship among task difficulty, sampling budget, verifier quality, and inference computation. DeepSeek-R1 demonstrated that reinforcement learning could incentivize sophisticated reasoning behavior, including reflection and verification, while s1 showed that carefully selected training examples combined with budget forcing could produce strong reasoning and controllable test-time computation. This review examines outcome and process reward models, reinforcement learning, search, self-correction, budget forcing, inference latency, reasoning-token allocation, and distillation. Remaining challenges include excessive reasoning, unreliable self-verification, reward hacking, evaluation contamination, hidden computational cost, safety degradation, and the difficulty of establishing whether extended reasoning traces correspond to faithful internal computation.
Scenario-Based Disaster Recovery Training for Improving Incident-Response and RTO/RPO Decision-Making Among Enterprise Cloud Engineers
Enterprise cloud disaster recovery depends not only on resilient architectures but also on engineers who can recognize incident scope, translate business impact into recovery objectives, select appropriate recovery strategies, and execute restoration under time pressure. Conventional policy review and lecture-based instruction provide limited opportunities to practice these coupled technical and judgment-intensive tasks. This review synthesizes evidence on scenario-based cybersecurity and disaster-recovery training and develops a competency-oriented framework for improving incident-response performance and recovery time objective (RTO)/recovery point objective (RPO) decision-making among enterprise cloud engineers. A structured review was conducted across peer-reviewed literature on cyber ranges, incident-response tabletop exercises, serious games, experiential cybersecurity training, and disaster-recovery evaluation, supplemented by authoritative contingency-planning and cloud-resilience guidance. Eligible evidence addressed realistic or simulated incident scenarios, measurable learning or performance outcomes, exercise design, or RTO/RPO-based recovery decisions, with findings synthesized across individual, team, and technical decision domains. The evidence consistently favors authentic, scenario-driven practice over passive knowledge transfer for developing incident-response awareness, procedural fluency, communication, and applied decision-making. Cyber-range studies indicate that realistic environments can support measurable acquisition of incident-response knowledge and skills, while tabletop and serious-game studies demonstrate benefits for preparedness, role clarity, strategic reasoning, and security self-efficacy. In disaster-recovery contexts, the principal contribution of training is not to alter the technical limits of replication or restore mechanisms, but to improve the selection and execution of recovery actions under explicit business constraints. Effective exercises therefore require workload-specific RTO and RPO targets, progressive incident injects, dependency failures, observable performance metrics, and structured debriefing. Scenario-based disaster-recovery training is therefore a credible approach for strengthening the socio-technical competencies required of enterprise cloud engineers. Programs are most defensible when they assess objective decision quality and operational performance rather than satisfaction alone, and when scenarios compel engineers to reconcile business impact, data-loss tolerance, recovery architecture, security containment, and service restoration.
Copyright (c) 2024 Journal of Computing, Intelligence and Information Sciences
Video-Capable Multimodal Large Language Models: Temporal Representation, Long-Video Memory, and Visual Instruction Tuning
Multimodal large language models initially focused primarily on static image-language interaction, but by 2024 increasing research attention had shifted toward video understanding. Video introduces temporal order, motion, event duration, repeated visual content, and substantially larger token requirements than individual images. This review examines the evolution of video-capable multimodal language models and long-video reasoning through 2024. Earlier approaches such as VideoBERT and contrastive video-language representation learning established foundations for joint visual-linguistic modeling. Large vision-language architectures including CLIP, Flamingo, BLIP-2, and LLaVA subsequently provided transferable visual-language components that could be adapted to video. Video-LLaMA extended language-model interaction to audio-visual inputs, while Video-ChatGPT introduced large-scale video instruction data and open-ended video conversation. Video-LLaVA unified image and video representation before projection into the language space. Chat-UniVi employed dynamic unified visual tokens, and LLaMA-VID compressed individual frames into context and content tokens to enable much longer videos. TimeChat introduced timestamp-aware features and temporal localization, whereas MovieChat used sparse memory mechanisms to support long-video understanding. MVBench provided a broad evaluation framework for temporal multimodal reasoning. The review compares frame sampling, visual token compression, temporal modeling, instruction tuning, long-video memory, visual-language projection, and multimodal evaluation. Limitations include temporal hallucination, inadequate event ordering, excessive visual-token counts, weak localization, benchmark contamination, and difficulty maintaining detailed representations across long videos.
Adaptive and Self-Reflective Retrieval-Augmented Generation for Grounded Large Language Models
Retrieval-Augmented Generation became an important architecture for reducing exclusive reliance on the parametric knowledge of large language models. Conventional RAG systems typically retrieve a fixed number of passages for every query and pass them to a generator regardless of whether retrieval is necessary, whether the retrieved evidence is relevant, or whether conflicting documents are present. Research through 2024 increasingly addressed these limitations through adaptive retrieval, hierarchical document representation, retrieval-aware fine-tuning, corrective retrieval, and self-reflection. This review traces the development from sparse information retrieval and dense passage retrieval to REALM, conventional RAG, Fusion-in-Decoder, RETRO, Atlas, Self-RAG, RAPTOR, Adaptive-RAG, Corrective RAG, and Retrieval-Augmented Fine-Tuning. Self-RAG integrates special reflection tokens that allow a language model to determine when retrieval is appropriate and to critique retrieved evidence and generated content. RAPTOR recursively clusters and summarizes documents to permit retrieval at multiple levels of abstraction. Adaptive-RAG varies retrieval strategy according to estimated query complexity, while Corrective RAG evaluates the quality of retrieved evidence before generation. RAFT trains models to identify relevant evidence and disregard distractor documents in domain-specific settings. This review compares retrieval triggering, retriever-generator integration, hierarchical indexing, evidence selection, domain adaptation, citation, factuality, latency, and evaluation. Persistent limitations include retrieval failure, stale corpora, contradictory evidence, citation inaccuracies, context overload, domain mismatch, and hallucinated synthesis despite access to relevant evidence.
Long-Context Large Language Models: Efficient Attention, Positional Extrapolation, Streaming Inference, and Memory Optimization
The context window determines how much information a large language model can process during an individual inference sequence. Expanding this window became a major research priority as language models were increasingly applied to long documents, books, source-code repositories, multi-turn conversations, and retrieval-intensive workflows. Standard Transformer self-attention incurs quadratic memory and computational costs with sequence length, while models trained on shorter sequences frequently fail to generalize to unseen positional ranges. This review examines methods for extending and efficiently processing long contexts through 2024. Transformer-XL introduced segment-level recurrence, while sparse-attention systems including Longformer and BigBird reduced attention complexity. Rotary positional embeddings and attention with linear biases offered alternative positional representations. FlashAttention and FlashAttention-2 improved exact attention efficiency through IO-aware GPU implementations. Positional interpolation, YaRN, and LongRoPE extended pretrained models beyond their original context lengths through modifications to positional encodings. StreamingLLM demonstrated that retaining initial attention-sink tokens alongside a rolling cache could support stable streaming inference. LongLLMLingua addressed another dimension of the problem by compressing prompts before inference. The review also examines key-value cache growth, positional degradation, retrieval within long contexts, attention sinks, memory bandwidth, long-context evaluation, and the difference between nominal context-window size and effective information utilization. Evidence available by 2024 showed that context length had become a systems, architecture, representation, and evaluation problem rather than a single model hyperparameter.
Sparse Mixture-of-Experts Architectures for Scaling Large Language Models: Routing, Specialization, and Computational Efficiency
Scaling large language models by increasing the number of parameters improves representational capacity but also substantially increases training and inference cost when all model parameters are activated for every input token. Sparse Mixture-of-Experts architectures address this limitation by conditionally routing tokens through subsets of specialized neural-network components. This review examines the evolution of Mixture-of-Experts language modeling through 2024, beginning with classical adaptive and hierarchical expert models and progressing toward sparsely gated neural networks, GShard, Switch Transformers, expert-choice routing, GLaM, sparse upcycling, Mixtral, and DeepSeekMoE. Modern sparse models separate total parameter capacity from active computational cost by using routing mechanisms that select a limited number of experts for each token. Mixtral demonstrated the practical feasibility of sparse expert activation within competitive open-weight large language models, while DeepSeekMoE emphasized fine-grained expert segmentation and shared experts to encourage stronger specialization. The review analyzes token-choice and expert-choice routing, auxiliary load-balancing losses, expert capacity, communication overhead, expert collapse, distributed training, inference memory, routing instability, and specialization. Particular attention is given to the relationship between the nominal parameter count of an MoE model and the much smaller subset of parameters activated during individual forward passes. Challenges include load imbalance, cross-device communication, expert redundancy, routing interpretability, memory requirements, and increasingly complex deployment infrastructure. By 2024, Mixture-of-Experts modeling had emerged as an important route toward increasing model capacity without proportionally increasing per-token computation.
Parameter-Efficient Adaptation and Quantization of Large Language Models: From Adapters and Prompt Tuning to LoRA and QLoRA
The rapid increase in the parameter count of pretrained language models created significant barriers to conventional task-specific fine-tuning. Updating every parameter of a multi-billion-parameter model requires substantial accelerator memory, storage, training time, and energy, while maintaining separate full model copies for multiple downstream tasks further increases deployment cost. Parameter-efficient fine-tuning seeks to adapt large pretrained models while modifying only a small fraction of their parameters. This review examines the evolution of efficient model adaptation through 2023. Adapter layers introduced compact trainable modules between frozen network layers, while BitFit demonstrated that useful adaptation could sometimes be achieved by updating only bias terms. Prefix tuning and prompt tuning shifted adaptation toward trainable continuous vectors inserted into the model's input or attention context. Low-Rank Adaptation introduced trainable low-rank decompositions of weight updates and became an increasingly influential approach for large language models. AdaLoRA dynamically allocated rank budgets according to parameter importance, while unified frameworks attempted to characterize common principles across multiple parameter-efficient methods. Alongside adaptation techniques, post-training and training-aware quantization reduced model memory requirements. LLM.int8() enabled large Transformer inference using mixed-precision decomposition, GPTQ provided accurate post-training weight quantization, and SmoothQuant shifted quantization difficulty from activations toward weights. QLoRA subsequently combined 4-bit quantized pretrained models with LoRA adapters, enabling efficient fine-tuning of very large language models on substantially reduced hardware. The review examines memory consumption, trainable parameter count, quantization error, downstream performance, optimizer requirements, inference overhead, and deployment trade-offs.
Copyright (c) 2023 Journal of Computing, Intelligence and Information Sciences
Controllable and Personalized Text-to-Image Diffusion Models: Spatial Conditioning, Subject Adaptation, and Semantic Image Editing
Diffusion-based text-to-image generation achieved substantial advances in visual quality and semantic expressiveness by 2022, but early systems provided users with comparatively limited control over spatial organization, identity preservation, object placement, and local image editing. Research during 2022 and 2023 increasingly shifted toward controllable and personalized diffusion systems capable of adapting large pretrained generative models without retraining them from scratch. This review examines major methods for controllable image synthesis available through 2023. Latent Diffusion Models established computationally efficient text-conditioned generation within compressed latent spaces, while classifier-free guidance supported flexible text conditioning. SDEdit demonstrated image transformation through stochastic differential equations, and ILVR introduced conditioning using reference-image information. Textual Inversion learned new concepts through optimized text embeddings, whereas DreamBooth adapted pretrained diffusion models to specific subjects from only a few reference images. Prompt-to-Prompt enabled semantic editing by manipulating cross-attention maps, while Null-text Inversion improved reconstruction and editing of real images. InstructPix2Pix introduced instruction-based image editing using paired synthetic supervision. Custom Diffusion reduced the parameter requirements of multi-concept personalization, while ControlNet provided explicit structural conditioning based on edges, depth maps, human poses, segmentation maps, and related signals. This review compares fine-tuning, embedding optimization, attention manipulation, inversion, structural conditioning, and plug-and-play approaches. Challenges include identity drift, concept overfitting, compositional failure, unauthorized personalization, memorization, computational cost, and reliable preservation of non-target image content.
Multimodal Large Language Models for General-Purpose Vision-Language Intelligence: Architectures, Instruction Tuning, and Emergent Capabilities
The convergence of large language modeling and vision-language pretraining led to rapid development of multimodal large language models during 2023. Earlier vision-language systems were typically optimized for predefined tasks such as captioning, retrieval, visual question answering, or recognition. Multimodal foundation models instead sought to connect pretrained visual encoders with large language models capable of instruction following and open-ended generation. This review examines the architectural evolution and principal capabilities of multimodal large language models available through 2023. Contrastive Language-Image Pretraining established transferable image-text alignment at scale, while Flamingo demonstrated few-shot multimodal learning by connecting frozen vision and language components. BLIP and BLIP-2 advanced bootstrapped vision-language pretraining, with BLIP-2 employing a lightweight Querying Transformer to bridge frozen image encoders and frozen large language models. LLaVA demonstrated visual instruction tuning using language-model-generated multimodal instruction data, while InstructBLIP systematically extended instruction tuning across diverse vision-language tasks. Concurrent systems including MiniGPT-4, Kosmos-1, PaLI, PaLI-X, and ImageBind explored different combinations of multimodal representation, language generation, and cross-modal transfer. The review analyzes visual encoders, connector modules, cross-attention, instruction tuning, frozen versus jointly trained components, multimodal benchmarks, and computational efficiency. Challenges identified in 2023 included visual hallucination, insufficient spatial reasoning, OCR limitations, weak fine-grained grounding, dataset contamination, safety, evaluation inconsistencies, and the difficulty of distinguishing genuine multimodal reasoning from language-prior exploitation.
Dense Information Retrieval and Retrieval-Augmented Language Models: Architectures for Grounded Knowledge-Intensive Artificial Intelligence
Although large pretrained language models encode substantial information within their parameters, purely parametric knowledge introduces limitations related to factual accuracy, knowledge freshness, provenance, and efficient updating. Retrieval-augmented architectures address these limitations by combining neural language models with external document collections. This review examines the evolution of dense retrieval and retrieval-augmented language modeling through 2023. Traditional sparse retrieval using term-frequency representations is considered as a foundation before examining neural semantic retrieval. Dense Passage Retrieval demonstrated that independently encoded queries and passages could support effective open-domain question answering through vector similarity. REALM incorporated retrieval into language-model pretraining, while Retrieval-Augmented Generation combined a pretrained sequence generator with retrieved documents treated as latent evidence. ColBERT introduced late interaction to preserve token-level matching while retaining retrieval efficiency. Fusion-in-Decoder improved generation over multiple retrieved passages, whereas RETRO demonstrated that retrieval could be integrated into very large language-model pretraining. Atlas further explored few-shot learning using retrieval-augmented language models. In 2023, Hypothetical Document Embeddings demonstrated zero-shot dense retrieval through language-model-generated pseudo-documents, while retrieval increasingly became central to grounding instruction-following systems. This review examines indexing, embedding models, vector similarity search, retriever training, reranking, document chunking, context fusion, factual grounding, retrieval latency, and knowledge updating. Challenges include retrieval errors, evidence conflicts, hallucinated synthesis, embedding-domain mismatch, stale indexes, evaluation methodology, and determining when retrieval should override parametric knowledge.
Effectiveness of Hands-On Lab Training for Improving Zero-Trust and Identity-Access Management Competencies Among Hybrid Cloud Engineers
Hybrid-cloud security increasingly depends on identity-centered controls, continuous authorization, federation, least privilege, and policy enforcement across heterogeneous platforms. These competencies are difficult to develop through lecture-only or awareness-oriented instruction because engineers must be able to configure, observe, troubleshoot, and validate security controls in realistic systems. This review evaluates the effectiveness of hands-on laboratory training for developing zero-trust and identity-access management (IAM) competencies relevant to hybrid-cloud engineering and proposes an evidence-based framework for training delivery and assessment. A structured evidence review synthesized peer-reviewed studies involving cybersecurity laboratories, virtual laboratories, cyber ranges, learner-controlled security training, gamified security education, and practical assessment, together with authoritative zero-trust and IAM standards. Evidence was mapped to operational competency domains including authentication, federation, role- and attribute-based authorization, least privilege, conditional access, workload identity, policy troubleshooting, and security telemetry. Across heterogeneous educational settings, practical training environments consistently supported procedural learning, realistic task execution, self-efficacy, engagement, and directly observable performance. Stronger training designs combined realistic scenarios with learner control, immediate feedback, repeatable environments, objective task instrumentation, and delayed or transfer assessment. Cyber-range and virtual-laboratory platforms also improved scalability and enabled systematic collection of command, log, configuration, and policy evidence. For zero-trust and IAM training, the most defensible outcome measures include policy correctness, task completion time, excess-privilege errors, authentication and federation troubleshooting accuracy, recovery from misconfiguration, and performance on unfamiliar scenarios. Overall, hands-on laboratories are well suited to hybrid-cloud zero-trust and IAM skill development because they convert abstract access-control principles into observable configuration and diagnostic behavior. Their effectiveness is greatest when training is scenario-based, instrumented, progressively scaffolded, and evaluated using task-level security outcomes rather than participant satisfaction alone.
Large Language Models and Instruction-Following Artificial Intelligence: Scaling, Alignment, Reasoning, and Tool-Augmented Learning
Large language models underwent a substantial transformation between large-scale autoregressive pretraining and the emergence of instruction-following conversational systems. By 2023, scaling model parameters, training data, and computational resources had produced increasingly general language models capable of in-context learning, reasoning, code generation, question answering, and instruction following. This review examines major developments underlying large language models through 2023. Transformer architectures and autoregressive language modeling are first considered as the technological foundation for model scaling. GPT-3 demonstrated strong few-shot and in-context learning behavior without conventional task-specific fine-tuning, while subsequent scaling-law research investigated relationships among model size, dataset size, training compute, and performance. Chinchilla scaling emphasized the importance of balancing model parameters with training-token volume. Instruction tuning subsequently improved generalization to previously unseen tasks, while reinforcement learning from human feedback provided a mechanism for aligning model outputs with human preferences. Chain-of-thought prompting, self-consistency, zero-shot reasoning, and ReAct demonstrated that prompting strategies could significantly influence reasoning behavior without modifying model parameters. Toolformer further explored self-supervised integration of external tools. The emergence of LLaMA and GPT-4 in 2023 demonstrated continuing improvements in general-purpose language understanding and generation. This review analyzes scaling, instruction tuning, human-feedback alignment, in-context learning, reasoning, tool use, evaluation, and safety. Remaining challenges included hallucination, bias, unreliable reasoning, prompt sensitivity, opaque training data, computational requirements, alignment uncertainty, and inadequate evaluation of rapidly emerging capabilities.
SVM-Based Disaster Recovery for Hybrid Enterprise Storage: Effects on Recovery Time and Service Availability
Hybrid enterprise storage combines on-premises infrastructure with geographically separated disaster-recovery or cloud-hosted resources, creating recovery requirements that extend beyond data replication alone. Effective recovery also depends on whether the storage namespace, protocol configuration, identity, and client-access state can be restored coherently following a site failure. Storage Virtual Machine (SVM)-based disaster recovery extends asynchronous replication from individual volumes to a logical storage service boundary containing multiple volumes and selected configuration objects. This review synthesizes evidence on the mechanisms through which SVM-level disaster recovery can influence recovery time and service availability and identifies the architectural and operational conditions required to realize these benefits in hybrid enterprise storage. A structured integrative review was conducted using evidence from enterprise storage, virtualization, high-availability, continuous data protection, wide-area migration, and cloud disaster-recovery literature, supplemented by technical documentation describing SVM replication behavior. Evidence was synthesized around recovery time objective (RTO), recovery point behavior, service availability, configuration preservation, network convergence, and operational failover complexity. The literature indicates that recovery delay is an end-to-end property. Asynchronous mirroring reduces production write-path overhead and enables tunable recovery points, while virtualized recovery mechanisms can reduce service interruption by pre-positioning execution or storage state. SVM-level protection provides an additional advantage by preserving a multi-volume storage namespace and substantial SVM-scoped configuration as a unified recovery domain, thereby reducing per-volume reconstruction requirements and configuration drift. Nevertheless, residual RTO remains dependent on failure detection, replication transition, destination activation, network and identity remapping, compute availability, protocol-specific host access, and application validation. Service availability therefore improves most when SVM replication is combined with rehearsed orchestration, continuous monitoring, compatible network design, and clearly bounded replication lag. Overall, SVM-based disaster recovery can materially reduce the operational component of RTO and improve service availability in hybrid storage environments, but replication alone is insufficient. Its principal benefit lies in recovering a coherent storage service boundary rather than a collection of independent data copies, with configuration state, network identity, compute dependencies, and validation treated as first-class components of the recovery path.
Copyright (c) 2022 Journal of Computing, Intelligence and Information Sciences
Transformer Architectures for Long-Horizon Time-Series Forecasting: Sparse Attention, Decomposition, and Frequency-Domain Modeling
Accurate long-horizon time-series forecasting is important in energy systems, transportation, finance, meteorology, industrial monitoring, healthcare, and capacity planning. Traditional statistical approaches remain effective in many structured settings, while recurrent neural networks and convolutional models introduced nonlinear representation learning for complex temporal sequences. Following the success of Transformers in natural-language processing, self-attention architectures were increasingly adapted for long-sequence forecasting. This review examines major developments in Transformer-based time-series forecasting through 2022. Conventional self-attention can represent long-range dependencies but exhibits quadratic computational complexity with respect to sequence length. LogSparse Transformer and Informer therefore introduced sparse attention strategies intended to reduce computational and memory requirements. Autoformer incorporated progressive seasonal-trend decomposition and an Auto-Correlation mechanism to model periodic dependencies at the subsequence level. FEDformer extended this direction by combining decomposition with frequency-enhanced representations based on Fourier and wavelet transformations. Pyraformer introduced pyramidal attention for multiresolution temporal relationships. These approaches are compared with recurrent forecasting, DeepAR, N-BEATS, Temporal Fusion Transformers, deep state-space models, and probabilistic multivariate forecasting. The review examines deterministic and probabilistic objectives, covariate integration, long-range dependency modeling, decomposition, computational complexity, and evaluation protocols. The 2022 literature demonstrated substantial interest in specialized temporal architectures but also highlighted concerns regarding benchmark consistency, sensitivity to forecasting horizon, model complexity, and whether increasingly elaborate attention mechanisms always outperform simpler forecasting baselines.
Neural Radiance Fields for Novel-View Synthesis and Neural 3D Scene Representation: Advances in Quality, Scale, and Rendering Efficiency
Neural scene representations rapidly transformed view synthesis and computer graphics following the introduction of Neural Radiance Fields. Rather than explicitly representing geometry with conventional meshes or point clouds, NeRF models optimize continuous neural functions that map spatial positions and viewing directions to volumetric density and emitted radiance. Combined with differentiable volume rendering, this formulation enables photorealistic novel views to be synthesized from collections of posed images. This review examines neural radiance-field research through 2022, focusing on improvements in image quality, scene scale, generalization, dynamic content, anti-aliasing, and rendering efficiency. The original NeRF formulation is reviewed together with positional encoding and related neural implicit representations. Mip-NeRF introduced integrated positional encoding to reduce aliasing, while NeRF-W addressed uncontrolled photographs containing appearance variation. PixelNeRF and IBRNet investigated generalization across scenes, and D-NeRF and Nerfies extended radiance fields to dynamic and deformable environments. By 2022, Mip-NeRF 360 expanded neural rendering to unbounded scenes, Mega-NeRF addressed large-scale environments, and Instant Neural Graphics Primitives dramatically reduced training time through multiresolution hash encoding. TensorRF and Plenoxels further demonstrated that explicit or factorized representations could offer substantial efficiency improvements compared with fully implicit multilayer perceptrons. The review analyzes rendering quality, training speed, memory consumption, scene representation, generalization, and scalability. Remaining challenges included sparse-view reconstruction, dynamic scenes, editing, geometry accuracy, relighting, and deployment on resource-constrained hardware.
Vision-Language Pretraining for Multimodal Artificial Intelligence: Contrastive Alignment, Cross-Modal Fusion, and Generative Learning
The integration of visual and linguistic information became a major direction in artificial intelligence by 2022. Earlier image-captioning and visual-semantic embedding systems typically relied on task-specific architectures and comparatively limited annotated datasets. Vision-language pretraining shifted this paradigm toward large-scale representation learning from paired image-text corpora, enabling models to transfer across image retrieval, visual question answering, image captioning, classification, grounding, and multimodal generation. This review examines major developments in vision-language representation learning through 2022. Early visual-semantic alignment and image-captioning approaches are reviewed as foundations for subsequent multimodal Transformers. Architectures such as ViLBERT, LXMERT, VisualBERT, UNITER, OSCAR, and ViLT introduced different strategies for fusing image-region representations and linguistic tokens. Contrastive Language-Image Pretraining demonstrated that large-scale image-text contrastive learning could produce transferable visual representations and strong zero-shot image classification. ALIGN extended contrastive pretraining to noisier web-scale image-text datasets, while ALBEF combined contrastive alignment with multimodal fusion. By 2022, BLIP introduced bootstrapped data filtering and unified understanding and generation, while Flamingo demonstrated few-shot multimodal learning through pretrained vision and language components. The review compares dual-encoder and fusion architectures, contrastive and generative objectives, region-based and patch-based visual inputs, zero-shot transfer, dataset scale, and computational requirements. Persistent challenges included noisy web supervision, multimodal hallucination, evaluation reliability, representational bias, fine-grained grounding, and the cost of increasingly large multimodal models.
Diffusion Models for High-Fidelity Image Synthesis: From Score Matching to Latent Generative Modeling
Deep generative modeling underwent a substantial transition by 2022 as diffusion-based approaches emerged as strong alternatives to generative adversarial networks, variational autoencoders, and autoregressive image models. Diffusion models formulate generation as the reversal of a gradual noise-corruption process, allowing complex data distributions to be learned through repeated denoising operations. This review examines the development of diffusion-based generative modeling through 2022, beginning with nonequilibrium thermodynamic formulations and denoising score matching and progressing toward denoising diffusion probabilistic models, score-based stochastic differential equations, deterministic implicit sampling, classifier guidance, and latent diffusion. Denoising diffusion probabilistic models demonstrated high-quality synthesis using parameterized reverse diffusion, while improved parameterizations and classifier guidance substantially enhanced sample fidelity. Score-based generative modeling unified several related approaches through stochastic differential equations. Denoising diffusion implicit models reduced sampling requirements by defining non-Markovian reverse processes. Subsequent classifier-free conditioning enabled controllable generation without a separate classifier. Latent diffusion further addressed the substantial computational cost of pixel-space models by performing the generative process within compressed perceptual representations. Concurrent text-conditioned systems illustrated the capacity of diffusion architectures to connect natural-language representations with high-resolution image synthesis. The review compares these methods in terms of likelihood formulation, sampling efficiency, conditioning mechanisms, image fidelity, latent representation, and computational requirements. Challenges remaining in 2022 included slow iterative inference, expensive large-scale training, evaluation of semantic alignment, dataset bias, memorization, and the computational accessibility of high-resolution generation.
Deep Reinforcement Learning for Computation Offloading and Dynamic Resource Allocation in Mobile Edge Computing
Mobile edge computing extends computational and storage capabilities toward the network edge to support latency-sensitive and resource-intensive applications running on mobile and Internet of Things devices. A central challenge is determining when computation should be processed locally, transferred to an edge server, or allocated across available network resources under rapidly changing wireless and computational conditions. Conventional optimization methods may become difficult to apply when system dynamics are unknown, high dimensional, or continuously varying. Deep reinforcement learning provides a framework in which agents learn sequential resource-management policies through interaction with the environment. This review examines deep-reinforcement-learning approaches to computation offloading and dynamic resource allocation available through 2021. Foundational value-based and actor-critic techniques - including deep Q-networks, asynchronous actor-critic methods, deterministic policy gradients, proximal policy optimization, and soft actor-critic - are reviewed in relation to edge-computing problems. Applications involving task offloading, wireless channel assignment, power management, vehicular edge computing, and multi-user resource allocation are subsequently examined. The review analyzes state-space design, reward formulation, discrete and continuous action spaces, single- versus multi-agent environments, convergence, energy consumption, service latency, and quality-of-service constraints. Persistent limitations include training instability, exploration cost, non-stationary network environments, generalization across deployment conditions, and the difficulty of validating learned policies in real infrastructure. Nevertheless, by 2021 deep reinforcement learning had emerged as an increasingly important mechanism for adaptive edge-resource orchestration.
Copyright (c) 2021 Journal of Computing, Intelligence and Information Sciences
RTO and RPO Optimization Using Asynchronous Replication in Multi-Petabyte Enterprise Storage Environments
Multi-petabyte enterprise storage platforms must sustain high write volumes while preserving geographically separated recovery copies. Although synchronous replication can constrain write latency and geographic distance, inadequately engineered asynchronous replication may accumulate backlog, widen the recovery point objective (RPO), and prolong recovery operations. This study synthesizes storage-systems and disaster-recovery literature to develop a practical framework for optimizing recovery time objective (RTO) and RPO in large enterprise environments using asynchronous replication. A structured narrative review and systems synthesis was conducted using peer-reviewed literature in storage systems, distributed systems, disaster recovery, and cloud computing. Evidence was examined across replication scheduling, write coalescing, consistency preservation, snapshot and checkpoint design, bandwidth provisioning, failure recovery, and multi-site orchestration. Across heterogeneous architectures, the principal determinants of effective RPO were replication interval, data-change rate, available replication bandwidth, queue growth, and remote-commit delay. RTO was influenced less by replication mode itself than by failure detection, storage promotion, consistency validation, application restart, network redirection, and recovery automation. At multi-petabyte scale, full-copy recovery is operationally inefficient, making bounded-delta transfer, changed-block tracking, consistency groups, parallel replication domains, and resumable resynchronization essential. A closed-loop optimization model is therefore proposed in which replication cadence and parallelism are continuously aligned with workload change rate and effective WAN throughput, while failover components are pre-staged and independently validated. Asynchronous replication can provide low operational overhead and predictable recovery at multi-petabyte scale when RPO is treated as a queue-stability problem and RTO as an orchestration problem. The most effective architecture separates data-protection cadence from service-recovery workflow, applies workload-specific protection classes, and validates recovery objectives through recurrent failover testing under representative peak conditions.
Tiny Machine Learning for Resource-Constrained IoT Devices: Efficient Architectures, Quantization, and Hardware-Aware Inference
The proliferation of low-power sensors and Internet of Things devices has created increasing demand for machine-learning inference directly on microcontrollers and other resource-constrained hardware. Conventional deep neural networks frequently exceed the memory, computation, latency, and energy budgets available on such platforms. Tiny machine learning, commonly termed TinyML, seeks to enable useful machine intelligence within kilobyte- to megabyte-scale memory environments while minimizing dependence on continuous cloud connectivity. This review examines the development of efficient embedded machine learning through 2021. Lightweight neural architectures including MobileNet, MobileNetV2, MobileNetV3, SqueezeNet, ShuffleNet, and EfficientNet are discussed together with model compression, pruning, low-precision inference, binary networks, quantization, neural architecture search, and hardware-aware optimization. Particular attention is given to MCUNet, which jointly designed neural architectures and an inference engine for microcontroller constraints, as well as Once-for-All and ProxylessNAS approaches for hardware-specialized deployment. Integer-only inference and mixed-precision quantization are evaluated as mechanisms for reducing memory and computational requirements. The review considers the trade-offs among model accuracy, SRAM consumption, flash storage, latency, energy efficiency, and device portability. Challenges include fragmented hardware ecosystems, limited benchmark standardization, memory bottlenecks, operator support, and differences between theoretical operation counts and real-device performance. The evidence available by 2021 suggested that hardware-software co-design would be central to making always-on artificial intelligence practical on very small IoT devices.
Self-Supervised Visual Representation Learning: Contrastive, Siamese, Clustering, and Self-Distillation Approaches
The dependence of supervised deep learning on large quantities of manually annotated data represents an important constraint in computer vision. Self-supervised learning addresses this limitation by constructing supervisory signals directly from unlabeled observations and learning transferable representations before downstream fine-tuning. Between 2018 and 2021, self-supervised visual learning progressed from handcrafted pretext tasks toward contrastive learning, momentum-based encoders, clustering, Siamese architectures, redundancy reduction, and self-distillation. This review examines the principal approaches available through 2021. Contrastive predictive coding and instance-discrimination techniques established the foundations for learning representations through similarity relationships. Momentum Contrast introduced dynamic dictionaries and momentum encoders, while SimCLR demonstrated the importance of data augmentation, projection heads, and large training batches. Bootstrap Your Own Latent showed that competitive representations could be obtained without explicit negative samples. SwAV combined clustering with swapped prediction between augmented views, whereas SimSiam demonstrated the importance of stop-gradient operations within comparatively simple Siamese networks. Barlow Twins approached representation learning through redundancy reduction, while DINO demonstrated strong emergent properties when self-distillation was combined with Vision Transformers. The review compares these approaches in terms of negative samples, architectural asymmetry, augmentation strategy, memory requirements, representation collapse, and transferability. By the end of 2021, self-supervised learning had become a major alternative to conventional supervised pretraining and provided an increasingly practical route for exploiting large unlabeled image collections.
Vision Transformers for Image Recognition and Dense Prediction: From Patch-Based Attention to Hierarchical Architectures
Convolutional neural networks dominated computer vision throughout much of the previous decade because their locality, translation equivariance, and hierarchical feature extraction provided effective inductive biases for image processing. By 2021, however, Transformer architectures originally developed for sequence modeling had become increasingly competitive alternatives for visual recognition. This review examines the development of Vision Transformers through 2021, beginning with the adaptation of self-attention to image patches and progressing toward more computationally efficient and hierarchical architectures. The Vision Transformer demonstrated that images could be represented as sequences of fixed-size patches and processed largely through standard Transformer encoders. Subsequent methods addressed the substantial data and computational requirements of the original architecture. Data-efficient image Transformers introduced attention-based knowledge distillation, while Tokens-to-Token ViT incorporated progressive tokenization to better represent local image structure. Pyramid Vision Transformer and Swin Transformer introduced hierarchical feature representations suitable for object detection and semantic segmentation. Convolutional Vision Transformer and LeViT further explored hybridization between convolutional inductive biases and attention mechanisms. This review compares these approaches in terms of token generation, self-attention strategy, computational complexity, hierarchical representation, data efficiency, and suitability for dense prediction. Major limitations involving quadratic attention complexity, training-data requirements, positional representations, and deployment efficiency are discussed. The evidence available by 2021 indicated that Transformers were becoming viable general-purpose computer-vision backbones rather than remaining exclusively language-processing architectures.