Technical Design: Smart Automation Engine
Technical design for manual ingestion, learned templates, OCR, PDF parsing, review sessions, and future domain extensions.
1. Overview
The Smart Automation Engine is the primary data entry mechanism for Homey. It eliminates manual typing fatigue by extracting item data from various sources using AI/ML.
MVP 1.0 Focus: Kitchen/Grocery domain (receipts, expiry dates)
Future Domains: Wardrobe (clothing), Toolbox (tools), Medicine Cabinet, etc.
1.1 Domain-Agnostic Architecture
The automation engine is designed to be extensible across domains:
Smart Automation Engine (Core)
├── Input Sources
│ ├── Camera (OCR, Object Detection)
│ ├── PDF/Document Parser
│ └── Manual Entry
├── Domain Recognizers (Pluggable)
│ ├── GroceryRecognizer (MVP 1.0)
│ │ ├── Receipt parsing (Vijetha, BigBasket)
│ │ └── Expiry date extraction
│ ├── ClothingRecognizer (Future)
│ │ ├── Wardrobe photo analysis
│ │ └── Garment type detection
│ └── ToolRecognizer (Future)
│ └── Toolbox photo analysis
└── Output → Domain-specific Items
├── GroceryItem (extends Item)
├── ClothingItem (extends Item)
└── ToolItem (extends Item)
1.2 MVP 1.0 Scope (Kitchen Domain)
This document focuses on the Kitchen/Grocery domain:
-
PDF Invoices - Digital receipts (BigBasket)
-
Paper Receipts - Camera OCR (Vijetha, Organic World)
-
Photo-to-Date - Capture expiry dates from packaging
Key principles: - Safety-First: Exact match policy prevents accidental item merging (allergy protection) - Ask Once: First scan prompts for details; subsequent scans use learned defaults - Staging Buffer: User reviews extracted items before committing to inventory
2. Feature Breakdown (Phases)
| Phase | Feature | Description | Priority |
|---|---|---|---|
1 |
Manual Quick-Add |
Fast manual entry with category/location presets |
HIGH |
2 |
Item Templates |
"Memory" system - learn defaults from first scan |
HIGH |
3 |
Camera OCR |
Paper receipt scanning with ML Kit |
HIGH |
4 |
PDF Parsing |
BigBasket invoice extraction |
MEDIUM |
5 |
Photo-to-Date |
Capture expiry date images |
LOW |
6 |
Staging Area |
Review/edit before committing batch items |
MEDIUM |
Recommended Order: 1 → 2 → 6 → 3 → 4 → 5
Rationale: Manual entry + templates provides immediate value. Staging area is needed before batch imports (OCR/PDF).
3. Data Model
3.1 New Entities
ItemTemplate (Learning System)
Stores learned defaults for recurring items.
@Entity(tableName = "item_templates")
data class ItemTemplateEntity(
@PrimaryKey
val id: String, // UUID
// Matching criteria (exact match required)
val canonicalName: String, // Normalized name for matching
val brand: String?, // Brand name (optional)
val variant: String?, // Variant details (optional)
// Learned defaults
val defaultCategory: String, // ItemCategory enum name
val defaultLocationId: String?, // Preferred storage location
val defaultExpiryDays: Int?, // Days from scan to expiry
val defaultUnit: String, // Default unit (kg, L, pack, etc.)
val defaultQuantity: Double, // Default quantity
// Metadata
val isCanonical: Boolean = false, // From pre-built taxonomy
val useCount: Int = 0, // Times this template was used
val createdAt: Date,
val lastUsedAt: Date?
)
ScanSession (Staging Area)
Tracks a batch scanning session before commit.
@Entity(tableName = "scan_sessions")
data class ScanSessionEntity(
@PrimaryKey
val id: String, // UUID
val sourceType: String, // PDF, OCR, MANUAL
val sourceName: String?, // "BigBasket Invoice", "Vijetha Receipt"
val status: String, // PENDING, COMMITTED, DISCARDED
val createdAt: Date,
val committedAt: Date?
)
@Entity(
tableName = "staged_items",
foreignKeys = [
ForeignKey(
entity = ScanSessionEntity::class,
parentColumns = ["id"],
childColumns = ["sessionId"],
onDelete = ForeignKey.CASCADE
)
]
)
data class StagedItemEntity(
@PrimaryKey
val id: String, // UUID
val sessionId: String, // FK to scan_sessions
// Extracted data
val rawText: String?, // Original text from OCR/PDF
val parsedName: String,
val parsedBrand: String?,
val parsedQuantity: Double,
val parsedUnit: String,
// User-confirmed/edited data
val confirmedName: String?,
val confirmedCategory: String?,
val confirmedLocationId: String?,
val confirmedExpiryDate: Date?,
val confirmedQuantity: Double?,
val confirmedUnit: String?,
// Status
val status: String, // PENDING, CONFIRMED, SKIPPED, ERROR
val matchedTemplateId: String?, // If matched to existing template
val needsReview: Boolean, // Requires user attention
val errorMessage: String? // Parse error if any
)
3.2 Database Migration
// Migration v2 -> v3
val MIGRATION_2_3 = object : Migration(2, 3) {
override fun migrate(database: SupportSQLiteDatabase) {
// Item templates table
database.execSQL("""
CREATE TABLE IF NOT EXISTS item_templates (
id TEXT PRIMARY KEY NOT NULL,
canonicalName TEXT NOT NULL,
brand TEXT,
variant TEXT,
defaultCategory TEXT NOT NULL,
defaultLocationId TEXT,
defaultExpiryDays INTEGER,
defaultUnit TEXT NOT NULL,
defaultQuantity REAL NOT NULL,
isCanonical INTEGER NOT NULL DEFAULT 0,
useCount INTEGER NOT NULL DEFAULT 0,
createdAt INTEGER NOT NULL,
lastUsedAt INTEGER
)
""")
// Create index for fast template matching
database.execSQL("""
CREATE INDEX idx_template_match
ON item_templates(canonicalName, brand, variant)
""")
// Scan sessions table
database.execSQL("""
CREATE TABLE IF NOT EXISTS scan_sessions (
id TEXT PRIMARY KEY NOT NULL,
sourceType TEXT NOT NULL,
sourceName TEXT,
status TEXT NOT NULL,
createdAt INTEGER NOT NULL,
committedAt INTEGER
)
""")
// Staged items table
database.execSQL("""
CREATE TABLE IF NOT EXISTS staged_items (
id TEXT PRIMARY KEY NOT NULL,
sessionId TEXT NOT NULL,
rawText TEXT,
parsedName TEXT NOT NULL,
parsedBrand TEXT,
parsedQuantity REAL NOT NULL,
parsedUnit TEXT NOT NULL,
confirmedName TEXT,
confirmedCategory TEXT,
confirmedLocationId TEXT,
confirmedExpiryDate INTEGER,
confirmedQuantity REAL,
confirmedUnit TEXT,
status TEXT NOT NULL,
matchedTemplateId TEXT,
needsReview INTEGER NOT NULL DEFAULT 0,
errorMessage TEXT,
FOREIGN KEY(sessionId) REFERENCES scan_sessions(id) ON DELETE CASCADE
)
""")
}
}
3.3 Pre-built Taxonomy (Starter Pack)
Seed data for common grocery categories with typical expiry durations:
object GroceryTaxonomy {
val STARTER_TEMPLATES = listOf(
// Dairy
Template("Milk", "DAIRY", expiryDays = 7, unit = "L"),
Template("Curd/Yogurt", "DAIRY", expiryDays = 14, unit = "kg"),
Template("Paneer", "DAIRY", expiryDays = 7, unit = "g"),
Template("Butter", "DAIRY", expiryDays = 30, unit = "g"),
Template("Cheese", "DAIRY", expiryDays = 30, unit = "g"),
// Produce
Template("Tomatoes", "PRODUCE", expiryDays = 7, unit = "kg"),
Template("Onions", "PRODUCE", expiryDays = 30, unit = "kg"),
Template("Potatoes", "PRODUCE", expiryDays = 21, unit = "kg"),
Template("Leafy Greens", "PRODUCE", expiryDays = 5, unit = "bunch"),
// Pantry
Template("Rice", "PANTRY", expiryDays = 365, unit = "kg"),
Template("Flour/Atta", "PANTRY", expiryDays = 180, unit = "kg"),
Template("Oil", "PANTRY", expiryDays = 365, unit = "L"),
Template("Sugar", "PANTRY", expiryDays = 730, unit = "kg"),
Template("Salt", "PANTRY", expiryDays = null, unit = "kg"), // No expiry
Template("Pasta", "PANTRY", expiryDays = 365, unit = "g"),
Template("Pulses/Dal", "PANTRY", expiryDays = 365, unit = "kg"),
// Beverages
Template("Tea", "BEVERAGES", expiryDays = 365, unit = "g"),
Template("Coffee", "BEVERAGES", expiryDays = 180, unit = "g"),
Template("Juice", "BEVERAGES", expiryDays = 7, unit = "L"),
// Meat
Template("Chicken", "MEAT", expiryDays = 3, unit = "kg"),
Template("Fish", "MEAT", expiryDays = 2, unit = "kg"),
Template("Eggs", "MEAT", expiryDays = 21, unit = "unit"),
// Frozen
Template("Frozen Vegetables", "FROZEN", expiryDays = 180, unit = "g"),
Template("Ice Cream", "FROZEN", expiryDays = 90, unit = "L"),
// Snacks
Template("Biscuits", "SNACKS", expiryDays = 90, unit = "g"),
Template("Chips", "SNACKS", expiryDays = 60, unit = "g"),
Template("Namkeen", "SNACKS", expiryDays = 60, unit = "g")
)
}
4. Phase 1: Manual Quick-Add
4.2 UI Design
+------------------------------------------+
| Quick Add Item [X] |
+------------------------------------------+
| |
| What are you adding? |
| [Search or type item name______] |
| |
| --- Suggestions (from templates) --- |
| [Milk] [Bread] [Eggs] [Tomatoes] |
| [Onions] [Rice] [+ More...] |
| |
| --- Or select category --- |
| [Dairy] [Produce] [Meat] [Pantry] |
| [Beverages] [Snacks] [Frozen] |
| |
+------------------------------------------+
| (After selection, show quick form) |
| |
| Milk |
| Brand: [Nandini_______________] |
| Quantity: [1____] [L v] |
| Expiry: [7 days from today v] |
| [Custom date...] |
| Location: [Fridge > Top Shelf v] |
| |
| [ ] Remember these defaults |
| |
| [Cancel] [Add Item] |
+------------------------------------------+
4.3 Interactions
| Action | Behavior | Result |
|---|---|---|
Type in search |
Filter templates by name |
Show matching suggestions |
Tap suggestion chip |
Auto-fill form with template defaults |
Form populated |
Tap category |
Show category-specific templates |
Filtered suggestions |
Check "Remember defaults" |
Save as new template on submit |
Template created |
Tap "Add Item" |
Create item + optionally template |
Item added to location |
4.4 Component Specification
@Composable
fun QuickAddScreen(
onItemAdded: (Item) -> Unit,
onDismiss: () -> Unit,
viewModel: QuickAddViewModel = hiltViewModel()
)
@HiltViewModel
class QuickAddViewModel @Inject constructor(
private val itemRepository: ItemRepository,
private val templateRepository: ItemTemplateRepository,
private val locationRepository: LocationRepository
) : ViewModel() {
val templates: StateFlow<List<ItemTemplate>>
val recentTemplates: StateFlow<List<ItemTemplate>>
val locations: StateFlow<List<Location>>
fun searchTemplates(query: String)
fun selectTemplate(template: ItemTemplate)
fun addItem(
name: String,
brand: String?,
category: ItemCategory,
quantity: Double,
unit: String,
expiryDate: Date?,
locationId: String,
rememberDefaults: Boolean
)
}
5. Phase 2: Item Templates (Learning System)
5.2 Template Matching Algorithm
object TemplateMatching {
/**
* Finds a matching template for the given item details.
* Uses exact match on normalized name + brand + variant.
*/
fun findMatch(
name: String,
brand: String?,
variant: String?,
templates: List<ItemTemplate>
): ItemTemplate? {
val normalizedName = normalize(name)
val normalizedBrand = brand?.let { normalize(it) }
val normalizedVariant = variant?.let { normalize(it) }
return templates.find { template ->
normalize(template.canonicalName) == normalizedName &&
(normalizedBrand == null || normalize(template.brand) == normalizedBrand) &&
(normalizedVariant == null || normalize(template.variant) == normalizedVariant)
}
}
/**
* Normalizes text for matching:
* - Lowercase
* - Remove extra whitespace
* - Remove common suffixes (1L, 500g, etc.)
*/
private fun normalize(text: String?): String? {
if (text == null) return null
return text
.lowercase()
.trim()
.replace(Regex("\\s+"), " ")
.replace(Regex("\\d+\\s*(kg|g|l|ml|pack|pcs|unit)s?$", RegexOption.IGNORE_CASE), "")
.trim()
}
}
5.3 New Item Flow
First scan of "Nandini Toned Milk 1L"
│
▼
Template found? ──No──> Show "New Item" dialog
│ │
Yes │
│ ▼
▼ User confirms:
Apply defaults - Category: Dairy
to staged item - Expiry: 7 days
- Location: Fridge
│
▼
Create template
for future scans
5.4 "New Item" Dialog
+------------------------------------------+
| New Item Detected [X] |
+------------------------------------------+
| |
| "Nandini Toned Milk 1L" |
| |
| We haven't seen this item before. |
| Set up defaults for future scans: |
| |
| Category: [ Dairy v ] |
| |
| Expiry: [ 7 days from scan v ] |
| ( ) Fixed date |
| ( ) No expiry |
| |
| Default Storage: |
| [ Fridge > Top Shelf v ] |
| |
| [Skip (manual each time)] [Save] |
+------------------------------------------+
5.5 Repository Interface
interface ItemTemplateRepository {
fun getAllTemplates(): Flow<List<ItemTemplate>>
fun getRecentTemplates(limit: Int = 10): Flow<List<ItemTemplate>>
fun searchTemplates(query: String): Flow<List<ItemTemplate>>
fun getTemplatesByCategory(category: ItemCategory): Flow<List<ItemTemplate>>
suspend fun findMatchingTemplate(
name: String,
brand: String?,
variant: String?
): ItemTemplate?
suspend fun createTemplate(template: ItemTemplate)
suspend fun updateTemplate(template: ItemTemplate)
suspend fun incrementUseCount(templateId: String)
suspend fun deleteTemplate(templateId: String)
suspend fun seedStarterTemplates()
}
6. Phase 3: Camera OCR
6.2 Technology Stack
-
ML Kit Text Recognition (on-device, privacy-first)
-
CameraX for camera preview and capture
-
Receipt-specific parsing for Vijetha, Organic World formats
6.3 Android Permissions
<!-- AndroidManifest.xml -->
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required="false" />
<!-- ML Kit Text Recognition -->
<meta-data
android:name="com.google.mlkit.vision.DEPENDENCIES"
android:value="ocr" />
6.4 Dependencies
# gradle/libs.versions.toml
[versions]
mlkit-text = "16.0.0"
camerax = "1.3.1"
[libraries]
mlkit-text-recognition = { module = "com.google.mlkit:text-recognition", version.ref = "mlkit-text" }
camerax-core = { module = "androidx.camera:camera-core", version.ref = "camerax" }
camerax-camera2 = { module = "androidx.camera:camera-camera2", version.ref = "camerax" }
camerax-lifecycle = { module = "androidx.camera:camera-lifecycle", version.ref = "camerax" }
camerax-view = { module = "androidx.camera:camera-view", version.ref = "camerax" }
6.5 OCR Pipeline
Camera Capture
│
▼
InputImage
│
▼
ML Kit TextRecognizer
│
▼
Raw Text Blocks
│
▼
Receipt Parser (store-specific)
│
▼
List<ParsedLineItem>
│
▼
Template Matching
│
▼
Staged Items (for review)
6.6 Receipt Parser Interface
interface ReceiptParser {
val supportedStores: List<String>
/**
* Attempts to parse receipt text into line items.
* Returns null if the receipt format is not recognized.
*/
fun parse(rawText: String): ParseResult?
}
data class ParseResult(
val storeName: String?,
val date: Date?,
val items: List<ParsedLineItem>,
val total: Double?,
val confidence: Float // 0.0 to 1.0
)
data class ParsedLineItem(
val rawText: String,
val name: String,
val brand: String?,
val quantity: Double,
val unit: String,
val price: Double?,
val confidence: Float
)
// Implementations
class VijethaReceiptParser : ReceiptParser { ... }
class OrganicWorldReceiptParser : ReceiptParser { ... }
class GenericReceiptParser : ReceiptParser { ... } // Fallback
6.7 UI: Camera Scanner Screen
+------------------------------------------+
| ← Scan Receipt |
+------------------------------------------+
| |
| +------------------------------------+ |
| | | |
| | | |
| | [Camera Preview] | |
| | | |
| | +------------------------+ | |
| | | Receipt Guide | | |
| | | Align receipt | | |
| | | within frame | | |
| | +------------------------+ | |
| | | |
| | | |
| +------------------------------------+ |
| |
| Tip: Hold steady for best results |
| |
| [Gallery] [ Capture ] [Flash] |
+------------------------------------------+
6.8 Scanning ViewModel
@HiltViewModel
class ScannerViewModel @Inject constructor(
private val textRecognizer: TextRecognizer,
private val receiptParsers: List<ReceiptParser>,
private val stagingRepository: StagingRepository,
private val templateRepository: ItemTemplateRepository
) : ViewModel() {
sealed class ScanState {
object Idle : ScanState()
object Scanning : ScanState()
data class Processing(val progress: Float) : ScanState()
data class Success(val sessionId: String, val itemCount: Int) : ScanState()
data class Error(val message: String) : ScanState()
}
val scanState: StateFlow<ScanState>
fun processImage(imageProxy: ImageProxy) {
viewModelScope.launch {
_scanState.value = ScanState.Scanning
try {
// 1. Run ML Kit OCR
val inputImage = InputImage.fromMediaImage(
imageProxy.image!!,
imageProxy.imageInfo.rotationDegrees
)
val result = textRecognizer.process(inputImage).await()
// 2. Parse receipt
val rawText = result.text
val parseResult = receiptParsers
.firstNotNullOfOrNull { it.parse(rawText) }
?: ParseResult(/* generic fallback */)
// 3. Match templates and create staging session
val session = stagingRepository.createSession(
sourceType = "OCR",
sourceName = parseResult.storeName
)
parseResult.items.forEachIndexed { index, item ->
_scanState.value = ScanState.Processing(
progress = index.toFloat() / parseResult.items.size
)
val template = templateRepository.findMatchingTemplate(
item.name, item.brand, null
)
stagingRepository.addStagedItem(
sessionId = session.id,
rawText = item.rawText,
parsedName = item.name,
parsedBrand = item.brand,
parsedQuantity = item.quantity,
parsedUnit = item.unit,
matchedTemplateId = template?.id,
needsReview = template == null
)
}
_scanState.value = ScanState.Success(session.id, parseResult.items.size)
} catch (e: Exception) {
_scanState.value = ScanState.Error(e.message ?: "Scan failed")
} finally {
imageProxy.close()
}
}
}
}
7. Phase 4: PDF Parsing
7.2 Technology
-
PDF parsing: Apache PDFBox (Android port) or iText
-
File picker: Android Storage Access Framework
7.3 Dependencies
[libraries]
# Option 1: PDFBox (simpler, lighter)
pdfbox-android = { module = "com.tom-roush:pdfbox-android", version = "2.0.27.0" }
# Option 2: iText (more powerful, larger)
itext-core = { module = "com.itextpdf:itext7-core", version = "7.2.5" }
7.4 BigBasket Parser
class BigBasketPdfParser : InvoiceParser {
override fun parse(pdfBytes: ByteArray): ParseResult? {
val document = PDDocument.load(pdfBytes)
val stripper = PDFTextStripper()
val text = stripper.getText(document)
document.close()
// BigBasket invoice format:
// - Header with order details
// - Table with: Item Name | Qty | Price
// - Footer with totals
val items = mutableListOf<ParsedLineItem>()
// Parse line items using regex patterns
val itemPattern = Regex(
"""(\d+)\.\s+(.+?)\s+(\d+(?:\.\d+)?)\s*(kg|g|L|ml|pcs|pack)?\s+₹?(\d+(?:\.\d+)?)"""
)
itemPattern.findAll(text).forEach { match ->
val (_, name, qty, unit, price) = match.destructured
items.add(ParsedLineItem(
rawText = match.value,
name = name.trim(),
brand = extractBrand(name),
quantity = qty.toDoubleOrNull() ?: 1.0,
unit = unit.ifEmpty { "unit" },
price = price.toDoubleOrNull(),
confidence = 0.9f
))
}
return if (items.isNotEmpty()) {
ParseResult(
storeName = "BigBasket",
date = extractDate(text),
items = items,
total = extractTotal(text),
confidence = 0.85f
)
} else null
}
}
8. Phase 5: Photo-to-Date Fallback
8.2 Implementation
// Add to ItemEntity or GroceryItemEntity
val expiryPhotoUri: String? // URI to captured photo
// Add to StagedItemEntity
val expiryPhotoUri: String?
8.3 UI: Date Capture
+------------------------------------------+
| Expiry Date |
+------------------------------------------+
| |
| [📅 Pick Date] [📷 Capture Photo] |
| |
| (If photo captured, show thumbnail) |
| +------------------+ |
| | [Photo of | [X] Remove |
| | expiry date] | |
| +------------------+ |
| |
+------------------------------------------+
9. Phase 6: Staging Area
9.2 UI Design
+------------------------------------------+
| ← Review Items [Commit All]|
+------------------------------------------+
| BigBasket • 12 items • Jan 23, 2026 |
+------------------------------------------+
| |
| ⚠️ 3 items need attention |
| |
| --- Needs Review (3) --- |
| |
| +------------------------------------+ |
| | [!] Nandini Milk 1L | |
| | Brand: Nandini | |
| | → New item - set defaults | |
| | [Configure] [Skip] | |
| +------------------------------------+ |
| |
| +------------------------------------+ |
| | [!] Unknown Item | |
| | Raw: "ORGN WLD HNYLMN 250G" | |
| | → Couldn't parse | |
| | [Edit Manually] [Skip] | |
| +------------------------------------+ |
| |
| --- Ready to Add (9) --- |
| |
| +------------------------------------+ |
| | ✓ Amul Butter 500g | |
| | Qty: 1 • Dairy • Fridge | |
| | Expires: Feb 23, 2026 | |
| | [Edit] [Remove] | |
| +------------------------------------+ |
| |
| ... (more items) |
| |
+------------------------------------------+
| [Discard All] [Commit All] |
+------------------------------------------+
9.3 Staging ViewModel
@HiltViewModel
class StagingViewModel @Inject constructor(
private val stagingRepository: StagingRepository,
private val itemRepository: ItemRepository,
private val templateRepository: ItemTemplateRepository
) : ViewModel() {
val session: StateFlow<ScanSession?>
val stagedItems: StateFlow<List<StagedItem>>
val itemsNeedingReview: StateFlow<List<StagedItem>>
val readyItems: StateFlow<List<StagedItem>>
fun loadSession(sessionId: String)
fun updateStagedItem(
itemId: String,
name: String,
category: ItemCategory,
quantity: Double,
unit: String,
expiryDate: Date?,
locationId: String
)
fun skipItem(itemId: String)
fun removeItem(itemId: String)
fun configureNewItem(
itemId: String,
category: ItemCategory,
expiryDays: Int?,
defaultLocationId: String,
createTemplate: Boolean
)
fun commitAll() {
viewModelScope.launch {
val items = readyItems.value
items.forEach { staged ->
// Create actual item
itemRepository.insertGroceryItem(...)
// Update template use count if matched
staged.matchedTemplateId?.let {
templateRepository.incrementUseCount(it)
}
}
stagingRepository.commitSession(session.value!!.id)
}
}
fun discardAll() {
viewModelScope.launch {
stagingRepository.discardSession(session.value!!.id)
}
}
}
10. Navigation & Entry Points
10.1 Ingest Hub (Unified Entry)
+------------------------------------------+
| Add Items |
+------------------------------------------+
| |
| +----------------+ +----------------+ |
| | | | | |
| | [Camera] | | [PDF] | |
| | Scan Receipt | | Import Invoice| |
| | | | | |
| +----------------+ +----------------+ |
| |
| +------------------------------------+ |
| | | |
| | [Quick Add] | |
| | Add items manually | |
| | | |
| +------------------------------------+ |
| |
| --- Recent Scans --- |
| [BigBasket • 12 items • Today] |
| [Vijetha • 5 items • Yesterday] |
| |
+------------------------------------------+
10.2 Navigation Graph
// Add to navigation graph
sealed class AutomationScreen(val route: String) {
object IngestHub : AutomationScreen("automation/hub")
object QuickAdd : AutomationScreen("automation/quick-add")
object Scanner : AutomationScreen("automation/scanner")
object PdfImport : AutomationScreen("automation/pdf-import")
object Staging : AutomationScreen("automation/staging/{sessionId}") {
fun createRoute(sessionId: String) = "automation/staging/$sessionId"
}
}
11. File Structure
app/src/main/kotlin/dev/sysout/homey/app/features/
└── automation/
├── data/
│ ├── ItemTemplateEntity.kt
│ ├── ItemTemplateDao.kt
│ ├── ItemTemplateRepository.kt
│ ├── LocalItemTemplateRepository.kt
│ ├── ScanSessionEntity.kt
│ ├── StagedItemEntity.kt
│ ├── StagingDao.kt
│ ├── StagingRepository.kt
│ └── LocalStagingRepository.kt
├── parsing/
│ ├── ReceiptParser.kt
│ ├── VijethaReceiptParser.kt
│ ├── OrganicWorldReceiptParser.kt
│ ├── GenericReceiptParser.kt
│ ├── BigBasketPdfParser.kt
│ └── TemplateMatching.kt
├── ui/
│ ├── IngestHubScreen.kt
│ ├── QuickAddScreen.kt
│ ├── QuickAddViewModel.kt
│ ├── ScannerScreen.kt
│ ├── ScannerViewModel.kt
│ ├── PdfImportScreen.kt
│ ├── StagingScreen.kt
│ ├── StagingViewModel.kt
│ └── components/
│ ├── CameraPreview.kt
│ ├── StagedItemCard.kt
│ ├── NewItemDialog.kt
│ └── TemplateChips.kt
└── GroceryTaxonomy.kt
12. Implementation Order
| Phase | Effort | Dependencies |
|---|---|---|
1. Manual Quick-Add |
1 week |
None |
2. Item Templates |
1 week |
Phase 1 |
6. Staging Area |
1 week |
Phase 2 |
3. Camera OCR |
2 weeks |
Phase 6, ML Kit, CameraX |
4. PDF Parsing |
1 week |
Phase 6, PDFBox |
5. Photo-to-Date |
0.5 week |
CameraX |
Total Estimated Effort: 6-7 weeks
13. Domain Extensibility Architecture
13.1 Item Type Hierarchy
The sealed Item class supports domain-specific extensions:
// Base item - domain agnostic
sealed class Item {
abstract val id: UUID
abstract val name: String
abstract val location: Location
abstract val gridPosition: GridPosition?
abstract val sortOrder: Int
abstract val photoUri: String?
abstract val domain: ItemDomain // NEW: Identifies the domain
}
enum class ItemDomain {
GROCERY, // Kitchen domain (MVP 1.0)
CLOTHING, // Wardrobe domain (Future)
TOOL, // Toolbox domain (Future)
MEDICINE, // Medicine cabinet (Future)
GENERAL // Uncategorized
}
// Domain-specific extensions
data class GroceryItem(...) : Item() {
override val domain = ItemDomain.GROCERY
val brand: String?
val expiryDate: Date?
val quantity: Double
val unit: String
val category: GroceryCategory // DAIRY, PRODUCE, etc.
}
data class ClothingItem(...) : Item() { // Future
override val domain = ItemDomain.CLOTHING
val size: String?
val color: String?
val material: String?
val category: ClothingCategory // SHIRT, PANTS, etc.
}
data class ToolItem(...) : Item() { // Future
override val domain = ItemDomain.TOOL
val brand: String?
val condition: String?
val category: ToolCategory // SCREWDRIVER, HAMMER, etc.
}
13.2 Domain-Specific Categories
Each domain has its own category enum:
// Kitchen domain (MVP 1.0)
enum class GroceryCategory {
DAIRY, PRODUCE, MEAT, FROZEN,
PANTRY, BEVERAGES, SNACKS, GENERAL
}
// Wardrobe domain (Future)
enum class ClothingCategory {
SHIRT, TSHIRT, PANTS, SHORTS, DRESS,
JACKET, SWEATER, UNDERWEAR, SOCKS,
ACCESSORIES, SHOES, GENERAL
}
// Toolbox domain (Future)
enum class ToolCategory {
SCREWDRIVER, HAMMER, WRENCH, PLIERS,
DRILL, SAW, MEASURING, FASTENERS,
ELECTRICAL, PLUMBING, GENERAL
}
13.3 Domain-Specific Recognizers
The automation engine uses pluggable recognizers:
interface DomainRecognizer {
val domain: ItemDomain
val supportedInputTypes: List<InputType>
/**
* Analyze input and extract items for this domain.
*/
suspend fun recognize(input: RecognizerInput): RecognitionResult
}
enum class InputType {
CAMERA_PHOTO, // Photo of items (wardrobe, toolbox)
CAMERA_DOCUMENT, // Document/receipt scan
PDF_DOCUMENT, // PDF file
MANUAL_ENTRY // User typing
}
// MVP 1.0: Grocery recognizer
class GroceryRecognizer @Inject constructor(
private val receiptParsers: List<ReceiptParser>,
private val textRecognizer: TextRecognizer
) : DomainRecognizer {
override val domain = ItemDomain.GROCERY
override val supportedInputTypes = listOf(
InputType.CAMERA_DOCUMENT,
InputType.PDF_DOCUMENT,
InputType.MANUAL_ENTRY
)
// ...
}
// Future: Clothing recognizer using ML Kit Object Detection
class ClothingRecognizer @Inject constructor(
private val objectDetector: ObjectDetector
) : DomainRecognizer {
override val domain = ItemDomain.CLOTHING
override val supportedInputTypes = listOf(
InputType.CAMERA_PHOTO,
InputType.MANUAL_ENTRY
)
// Analyze wardrobe photo → detect shirts, pants, etc.
}
13.4 Domain-Aware Templates
Templates are scoped to their domain:
@Entity(tableName = "item_templates")
data class ItemTemplateEntity(
@PrimaryKey val id: String,
val domain: String, // ItemDomain enum name
val canonicalName: String,
// ... domain-agnostic fields ...
// Domain-specific defaults stored as JSON
val domainDefaults: String? // JSON blob for domain-specific fields
)
// Example domainDefaults for Grocery:
// {"expiryDays": 7, "category": "DAIRY", "unit": "L"}
// Example domainDefaults for Clothing:
// {"category": "SHIRT", "defaultSize": "M"}
13.5 Future Domain: Wardrobe Example
User Flow: 1. User opens wardrobe and takes a photo 2. App uses ML Kit Object Detection to identify garments 3. Staging area shows detected items: "3 shirts, 2 pants, 1 jacket" 4. User confirms/edits and commits to inventory
Technical Approach: - ML Kit Object Detection with custom model trained on clothing - Or: Google Cloud Vision API for image labeling - Or: Manual tagging with smart suggestions
13.6 Migration Path
Phase 1 (MVP 1.0): Kitchen domain only
- ItemDomain.GROCERY is the only active domain
- All existing items default to GROCERY domain
Phase 2: Add domain column
- Database migration adds domain column with default GROCERY
- UI starts showing domain selector in location settings
Phase 3: Additional domains - Add ClothingItem, ToolItem entities - Add domain-specific recognizers - Location can be scoped to a domain (Kitchen → GROCERY, Bedroom → CLOTHING)
14. Open Questions
-
Offline First: Should OCR work fully offline? (ML Kit supports both)
-
Store Support: Which stores to prioritize after Vijetha/Organic World?
-
Barcode Scanning: Add barcode/UPC lookup for packaged goods?
-
Cloud Templates: Sync templates between household devices?
-
Receipt Storage: Keep photos of receipts for warranty/return purposes?
-
Domain Detection: Auto-detect domain from location? (Kitchen → Grocery, Bedroom → Clothing)
-
Cross-Domain Items: Can an item exist in multiple domains? (e.g., cleaning supplies)
Document Version: 1.1 Created: 2026-01-23 Updated: 2026-01-23 - Added domain extensibility architecture Status: Ready for Review