Demystifying Archive Forms in BMC Remedy AR System: A Deep Dive

In the world of IT Service Management (ITSM) and enterprise applications, data management is paramount. BMC Remedy AR System, a powerful platform for building and managing ITSM solutions, is no exception. As your AR System environment grows, so does the volume of data. This is where the concept of Archive Forms becomes crucial. Archiving isn’t just about deleting old data; it’s a strategic approach to maintaining system performance, reducing storage costs, and ensuring compliance, all while keeping historical records accessible when needed.

This article will explore the intricacies of Archive Forms within BMC Remedy AR System, covering their purpose, creation, management, and the underlying technical concepts that make them work. We’ll delve into practical scenarios, common questions, and best practices to help you effectively leverage archiving in your AR System environment.

The “Why” Behind Archiving Forms

Imagine your AR System instance has been operational for several years. Ticket data, incident records, change requests – they all accumulate. Without a proper archiving strategy, this ever-increasing dataset can lead to several challenges:

• Performance Degradation: Larger databases mean longer query times, slower form loads, and a generally sluggish user experience. This impacts productivity and user satisfaction.
• Increased Storage Costs: Storing vast amounts of historical data on primary production databases can be expensive, especially for organizations with strict data retention policies.
• Backup and Restore Time: Larger databases take significantly longer to back up and restore, increasing downtime during maintenance or disaster recovery scenarios.
• Complexity in Reporting: Running reports on massive datasets can be slow and cumbersome, making it difficult to extract timely and actionable insights.
• Compliance Requirements: Many industries have regulations that mandate data retention for specific periods. Archiving ensures this compliance without burdening the production system.

Archive Forms offer a solution by creating separate, optimized storage for historical data. This allows your production environment to remain lean and performant, while still providing access to archived records for auditing, reporting, or historical analysis.

Understanding Archive Form Fundamentals

At its core, an Archive Form in AR System is essentially a copy of a production form, designed to store older data. When a record meets the defined archiving criteria, it’s moved from the production form to its corresponding archive form.

Key Concepts and Terminology

Before diving into the mechanics, let’s clarify some key terms:

• Archive Form: A separate form specifically created to store historical data from a production form.
• Archiving Criteria: The rules that determine when a record should be moved from the production form to the archive form (e.g., based on status, creation date, resolution date).
• Archiving Process: The mechanism or workflow that automatically identifies and moves records meeting the criteria. This is often implemented using Escalations.
• Data Retention Period: The length of time data is kept in the archive before it might be purged (depending on organizational policies).

The Technical Foundation: Database Tables

When AR System processes data, including archiving, it interacts with underlying database tables. Understanding these tables provides a deeper insight into how AR System manages your information.

Based on the provided information, here are some key tables and their roles:

• T (Transaction Table): This is the primary table that holds the actual data for your forms. When a record is archived, its data moves from the production T table to the archive form’s T table.
• B (Binary Table): This table stores binary data such as attachments and active links. When records with attachments are archived, these attachments are also moved, and their metadata is managed within this table.
• H (History Table): This table is crucial for tracking the history of changes to a record, especially status changes. When a ticket’s status history is significant, this table stores that granular detail. The “Status History” field (with Database ID 15) often leverages this table.

Furthermore, specific tables are involved in managing form and field definitions, and these play a role during installation and configuration:

• control: Often the first table created during AR System installation, it holds core system control information.
• controlRecordIds: Likely manages unique identifiers for control records.
• arschema: Stores the definitions of all forms (schemas) in the AR System.
• schema_index: Manages indexing for forms to improve query performance.
• schema_group_ids: Relates forms to group permissions.

Form Types and Their IDs

AR System categorizes forms into different types, each with a unique identifier:

• Regular Form (ID: 1): The standard form for most transactional data.
• Display Form (ID: 4): Used for displaying data, often in conjunction with Join Forms.
• Join Form (ID: 2): Allows you to combine data from multiple regular forms into a single view. BMC recommends joining up to 6 forms, though technically more are possible. The join criteria, once defined, cannot be changed in version 8.1.
• View Form (ID: 3): Represents a “view” of a form for different users, allowing customized data presentations.
• Vendor Form (ID: 5): Enables AR System to access external data sources through specific vendor integrations.

Creating and Managing Archive Forms

Archiving in AR System is typically a planned process, often involving the creation of an archive form and the setup of workflows to move data.

The Archiving Process: Step-by-Step

1. Define Archiving Criteria: Determine precisely which records should be archived. This is often based on fields like ‘Status’ (e.g., ‘Closed’, ‘Resolved’, ‘Cancelled’) and ‘Last Modified Date’ or ‘Completion Date’.
2. Create the Archive Form: This is a crucial step. You’ll essentially create a new form that mirrors the structure of the production form you want to archive. It’s good practice to name it clearly, e.g., “HPD:Help Desk – Archive”. The Archive Form inherits the Database ID of the original form when created, facilitating data transfer.
3. Develop Archiving Workflow: This is typically done using an Escalation. The escalation will periodically run, query the production form for records matching the archiving criteria, and then:
   • Copy the matching records to the archive form.
   • Delete the records from the production form.
4. Implement Data Retention and Purge Policies: Decide how long data should remain in the archive. You might set up another escalation to purge old data from the archive form after a specified period, ensuring that the archive itself doesn’t grow indefinitely.

Important Considerations:

• You cannot archive an archive form. This is a fundamental limitation.
• Archive forms are created immediately, but data transfer occurs based on the escalation schedule.
• Auditing of audit forms is not allowed. However, you can audit archive forms.
• Audit forms themselves can be archived.

License Management and User Access

Understanding license types is critical for managing user access in AR System. The question about fixed vs. floating licenses highlights this.

Fixed vs. Floating Licenses

Let’s clarify how licenses work, especially in scenarios with mixed license types:

• Fixed Licenses: These are assigned to specific users. A user with a fixed license can log in and use AR System at any time, as their license is “fixed” to them.
• Floating Licenses: These are pooled licenses available to any user. When a user logs in and requires a floating license, one is consumed from the pool. If all floating licenses are in use, subsequent users must wait until a license becomes available (e.g., when another user logs out).

Scenario: 5 Fixed and 5 Floating Licenses

With 5 fixed licenses, those 5 users can always log in and use the system. The 5 floating licenses add capacity. Therefore, up to 10 users can actively use the AR System at any given time, provided those 5 floating licenses are available. If more than 10 users try to log in simultaneously, those beyond the 10 will have to wait for a floating license to be released. Unlimited users can technically “log in” and wait for license availability.

Cost and Usage: Floating licenses are indeed often more costly than fixed licenses. Their “on-demand” nature makes them ideal for environments with shift workers or where user demand fluctuates significantly, ensuring that licenses are utilized efficiently during peak times without over-provisioning for every potential user.

Data Storage and Field Types

AR System has specific ways of handling various data types, including dates, times, and special fields.

Date/Time Storage

AR System stores date/time values as the integer number of seconds since January 1, 1970, 00:00:00 GMT. This epoch-based system allows for precise timestamps. This format is generally supported through January 18, 2038, marking the end of the 32-bit signed integer range for time representation (the “Year 2038 problem”).

Core Fields

Every AR System form has a set of core fields that are fundamental to its operation. Here are some of the most important:

• Request ID (1): The unique identifier for each record. This is auto-generated and crucial for referencing tickets. You can assign a prefix and restrict its input length via field properties.
• Submitter (2): The user who originally created the record.
• Create Date (3): The date and time the record was created.
• Assigned To (4): The current user or group assigned to the record.
• Last Modified By (5): The user who last modified the record.
• Modified Date (6): The date and time the record was last modified.
• Status (7): The current state of the record (e.g., New, Assigned, Resolved, Closed).
• SD (8): This often refers to a Short Description field.
• Status History (Hidden, 15): A hidden field that tracks the history of status changes for a record. You can view this via the “View Menu” -> “Status History” in Remedy User, or by pressing Ctrl+H in Modify mode in the User Tool.

Character vs. Diary Fields

A common point of confusion is the difference between Character and Diary fields:

• Character Fields: These are standard alphanumeric fields. Data entered into a character field can be overwritten. If you save a multi-byte character string exceeding the maximum size, you’ll encounter an ARERR [559] error.
• Diary Fields: These fields are designed for appending information. When you add data to a diary field, it’s appended along with the date, time, and the user who made the entry. This provides a chronological log of all entries, ensuring that all values are maintained. The database format after modification typically appears as [timestamp in seconds -| User -| Data], with subsequent entries appended.

Web Report Limitations: It’s important to note that Web reports generally do not support diary fields, attachment fields, or attachment pools. They also have limitations with currency values and types.

Attachment Pool

Attachments in AR System are managed through an Attachment Pool. When you attach a file, AR System creates:

• Attachment Field (Data Type: Attachment): The field on your form where you see and interact with the attachment.
• Property Field (Max Size: 0): This may refer to a field controlling attachment properties, with a default maximum size of 0, indicating no inherent limit set at the field level.

Two tables are involved in managing attachments:

• BSchema_id: This table stores metadata related to attachments for a specific form (schema). For each attachment, it creates three columns in the binary table: one depicting the file path, another the original file size, and a third the compressed file size (all in bytes).
• Battachpoolid: This table stores the actual binary data of the attachments, linked to the Request ID of the record they are associated with.

Menus: Static and Dynamic

Menus in AR System are powerful tools for providing users with selection lists. They can be:

• Static Menus:
  • Character: A predefined list of character strings.
  • Form data dictionary: Lists fields from a specified form.
  • Field data dictionary: Lists fields within a specific form.
• Dynamic Menus:
  • Search: Dynamically retrieves data based on search criteria.
  • SQL: Executes an SQL query to populate the menu.

File menus can be of both static and dynamic types.

Menu Refresh Rate: For all menu types except character menus (which are static), you must set a refresh mode:

• On Connect: Retrieves the menu when the user opens the menu after selecting the form. Reopening the form is needed for updates.
• On Open: Retrieves the menu every time the user opens it. Use judiciously to avoid performance impacts. For browsers, this acts like “On Open.”
• On 15 Minute Interval: Retrieves the menu upon first opening and then every 15 minutes. This balances currency with performance.

Menu definitions are updated every time you reconnect to a form. Importantly, deleting a menu-associated field or form will not delete the menu itself.

Menu Permissions: You cannot directly assign permissions to menus. However, the permissions applied to the character field to which the menu is attached will govern access to that menu.

Menu Levels and Children: Character and file menus support up to 15 levels and 99 children per level.

Groups and Permissions

Access control in AR System is managed through groups and roles. Understanding group types and categories is vital for effective security management.

Types of Groups

• Implicit Groups: These are system-defined groups with inherent permissions. Examples include public (0), Submitter (3), and Assignee (4).
• Explicit Groups: These are custom-defined groups created by administrators. Examples include Administrator (1), customize (2), and Sub Administrator (5).

Group Categories and Ranges

Groups are also categorized by their function and numerical range:

• Regular (1000 – 14999): Typically explicit groups used for broad access control.
• Computed (1000 – 14999): These groups might have dynamically determined members, often still falling under explicit management.
• Dynamic (60000 – 60999, e.g., 60001, 60002): These are implicit groups whose membership is determined by system rules or conditions, often related to roles or hierarchies.

Implicit Group Examples and Their “Type”

The provided list shows some implicit groups and their perceived “type” (though some might be miscategorized in the input data):

• public (0): Implicit
• Administrator (1): Explicit
• customize (2): Explicit
• Submitter (3): Implicit
• Assignee (4): Implicit
• Sub Administrator (5): Explicit
• Assignee Group (7): Implicit
• Struct Admin (8): Explicit
• Sub Struct Admin (9): Explicit

To allow a user other than “Demo” to log in to Developer Studio, you typically add them to a group like “Struct Sub-Admin” or a similar administrative group with the necessary permissions.

Giving Common Permissions to Forms

Manually assigning permissions to individual forms can be tedious. AR System provides a more efficient way:

1. Identify the group you want to grant permissions to.
2. Right-click on the group name.
3. Select “Assign permissions.”
4. Add the desired forms to that group’s permissions.

Workflow Automation in AR System

Workflows are the backbone of AR System’s automation capabilities, enabling dynamic and responsive processes.

What is Workflow?

Workflow can be defined as the set of processes and rules that govern how your company operates. In AR System, workflow automates these processes through components like Active Links, Filters, and Escalations, triggering actions in response to predefined conditions.

Workflow Components:

• Forms (Schema): These are the fundamental objects that represent data structures, analogous to tables in a database.
• Active Links: These workflows execute based on client-side operations or information displayed on the current screen. They are triggered by user interactions within the application. Examples include displaying messages, changing field label colors, or pushing/fetching data between forms.
• Filters: Filters execute when form transactions occur on the server (e.g., Submit, Modify, Delete operations). They can perform actions based on the data being processed.
• Escalations: Similar to filters, but they execute at predetermined time intervals. They check requests in the database for conditions that need to be met. This is the primary tool for implementing scheduled tasks like archiving.

Access Control

Permission Groups and Roles are used to control access to various AR System objects, including forms, rows (records), and columns (fields).

Developer Studio and Overlays

BMC Remedy Developer Studio is the primary tool for customizing and developing AR System applications. The concept of Granular Overlays is a key feature that has evolved to improve the upgrade and patching process.

Granular Overlays Explained

Granular overlays allow you to apply customizations to specific subcomponents of an object rather than the entire object. This means you can modify only certain aspects of an out-of-the-box (origin) object while inheriting other aspects from it. This approach significantly minimizes the effort required to reconcile customizations after upgrades or patches.

Types of Granular Overlays:

1. Additive Overlay (Extensions Overlay): Used when you want to add new features or information to an existing origin object. If the origin object changes during an upgrade, your additions are appended to the new definition.
2. Overwrite Overlay: This type completely replaces the origin object’s behavior for the overlaid components. The origin object’s definition is ignored for those specific parts. This is useful for scenarios like removing permissions or fundamentally changing an object’s behavior.
3. No Overlay (Inheritance Overlay): This is the default. If you don’t modify an object or its parts, they inherit their properties directly from the origin object. This ensures that out-of-the-box functionalities are retained unless explicitly overridden.

Installation and Configuration Details

Understanding where AR System stores its configuration and log files is crucial for troubleshooting and administration.

Key File and Folder Locations:

• AR System Installation Log Files: Typically found in C:\Users\Administrator\AppData\Local\Temp\1\arsystem_install_log.txt (the exact path may vary based on your Windows version and installation options).
• Hosts File and Services with Port Numbers: Essential for network communication and service identification. Located at C:\Windows\System32\drivers\etc.
• AR Database User and Database Name: By default, the database user is ARAdmin and the database is named Arsystem. The default password for ARAdmin is typically AR#Admin#.
• ARSYS Database Location: For Microsoft SQL Server, it’s often found in C:\Program Files (x86)\Microsoft SQL Server\MSSQL10_50.MSSQLSERVER\MSSQL\DATA (again, paths are illustrative and depend on your SQL Server installation).
• Server Names: Server names are stored within the workspace in a file called ARLogin.

Supported Technologies (ARS 8.1 Example):

Knowing the supported platforms is essential for planning and deployment:

• Operating Systems: Windows, Linux.
• Web Servers (for Mid Tier): Tomcat, Jboss, Servlet:Exec, IIS (though BMC doesn’t officially recommend IIS).
• Databases: IBM DB2, Oracle, Microsoft SQL.

Network Ports:

• Java Plug-in Server TCP Port: 9999
• Flash Board Port: 9998 (This port might require reconfirmation based on specific configurations).
• SMTP Port: 25

Troubleshooting Common Scenarios

Understanding potential issues and how to address them is part of effective AR System administration.

Status History Visibility

If you can’t see the “Status History” field (the 9th field), here’s how to access it:

• From Remedy User, modify the Status field of a ticket.
• Go to the “View” menu and select “Status History.”
• Alternatively, in the User Tool, open a form in modify state and press Ctrl+H.

Request ID Prefix and Length Restriction

To give a prefix (e.g., “ABC”) to the Request ID field and restrict its length:

1. Open the Request ID field in properties.
2. Set the Default Value to your desired prefix (e.g., ABC).
3. To restrict length, navigate to the Database tab in the field properties and set the Input Length.

Help Text Display

The display of help text in a field can vary based on the user’s login:

• Demo User: Will see the Field Name (label), Field ID, and the Help Text value.
• Other Registered Users: Will see only the Help Text value. If no Help Text is defined, it will show field description elements like symbols, keywords, or length.

Form Deletion and Workflow

When a form is deleted, any associated Active Links will also be deleted, provided that all active links associated with that form are removed.

Conclusion

Archive Forms are a fundamental component of a robust BMC Remedy AR System architecture. By strategically managing historical data, you can ensure optimal system performance, reduce operational costs, and meet compliance requirements. Understanding the underlying database structures, workflow automation, license management, and the nuances of field types and permissions empowers administrators to implement effective archiving strategies.

The principles discussed here, from license management to granular overlays and workflow components, are essential for any AR System administrator or developer. By mastering these concepts, you can build and maintain a more efficient, scalable, and reliable ITSM platform.