Creating Generators

Optic Generators provide a programmatic interface for working with different kinds of code. When you create a generator with the SDK, you provide Optic with enough information to learn how to parse, mutate and generate a certain kind of code.

An Optic generator for import statements might handle code of this form: const name = require('path/to/value').

  1. When parsing, Optic returns a model like this {definedAs: 'name', path: 'path/to/value'}
  2. When mutating, Optic patches existing code to match an updated model provided by the user. {definedAs: 'newName', path: 'path/to/new'} -> const newName = require('path/to/new')
  3. When generating, Optic takes a new model and writes code {definedAs: 'namevalue', path: 'optic'} -> const namevalue = require('optic')

We like to think of each Optic generator as an API for a certain kind of code. You can Get (parse), Post (generate), and Put (mutate) any code in your repo that has a corresponding Optic generator.

To get started, you just need to decide what kind of code you want to a generator for and decide on an appropriate abstraction. For instance, if I wanted to create an Optic generator for my Express JS endpoints, it would make sense to use a data model like {url: String, method: Enum, parameters: [Parameter], responses: [Response]} because when developers talk about an API, it's usually at that level of abstraction.

Example Snippet

Every Optic generator begins with an example code snippet that Optic can use as a template. So in the above example, teaching Optic to work with API endpoints, we would start off by 1) importing the js parser and then 2) providing a basic example:

import {js} from 'optic-skills-sdk'

js`
app.get('url', (req, res) => {

})`
.name('Express Endpoint')
.id('express-endpoint') 

The Abstraction

We now need an abstract model that describes the example snippet. Since we're building a generator for an API endpoint we know we'll need a method and a url field so we'll start with those.

import {js, literalWithValue, tokenWithValue} from 'optic-skills-sdk'

js`
app.get('url', (req, res) => {

})`
.name('Express Endpoint')
.id('express-endpoint')
.value({
	method: tokenWithValue('get'),
	url: literalWithValue('url'),
})

Setting up an abstraction for a generator is done by constructing an object with a certain shape. You'll see that each field is assigned a value using the finder methods from by the SDK. A finder will look for a certain value in the example snippet, and learn to associate the AST node it's found in with that field.

example relationship

These are the basic finders availible in the SDK:

Basic Finders usage returns
tokenWithValue('string', {options}) finds the token with value 'string' string
literalWithValue(bool string number, {options})
arrayWithValue([], ...{options}) finds the array literal with value [ any ]
objectWithValue({}, ...{options}) finds an object literal with value {any}

options: 1) occurrence: Number - when multiple finder matches are found, occurrence lets you choose the one you want in the order they appear in the example snippet.

So now when we give Optic the following examples, it knows how to parse them:

app.get('url', (req, res) => {  //interpreted as {"method": "get", "url": "url"}

})

app.put('/my/url', (req, res) => {  //interpreted as {"method": "put", "url": "my/url"}

})

We can make this work for parameters and responses by stacking Optic generators on top of one another. This is done using more advanced finders.

Advanced Finders usage returns
collect(abstractionOrGenerator) finds all instances (a, a, b, c) of abstraction / generator and adds them to an array [ abstraction ]
collectUnique(abstractionOrGenerator) finds all unique (a, b, c) instances of abstraction / generator and adds them to an array [ abstraction ]
mapToObject(abstractionOrGenerator, keyField) finds all instances of abstraction / generator and add them to an object using takes field 'keyField' as the key {key: abstraction, ...}

Once we add the fields for parameters and responses, Optic will be able to understand a more complex example.

.value({
	method: tokenWithValue('get'),
	url: literalWithValue('url'),
	parameters: collectUnique(parametersGenerator),
	responses: collectUnique('optic:rest/response'),
})
app.get('/hello-world', (req, res) => {  
	const firstName = req.query.firstName
	const lastName = req.query.lastName
	res.send(`Hello ${firstName} ${lastName})
})

/* Parsed by Optic as 
{
	"method": "get",
	"url": "hello-world",
	"parameters": [
		{"in": "query", "name": "firstName"},
		{"in": "query", "name": "lastName"},
	],
	"responses": [
		{"code": 200, "type": "string"}
	]
}
*/

You can check out a complete implimentation of this generator and its spec.

Other Options

Variables

When you define a variable for your generator, it tells Optic that a token can have any value, as long as it is consistent throughout this section of code.

For instance without a variable defined, all 3 instances of "result" must equal "result" for Optic to match this section of code.

function test(result) {
	message.post(result)
	system.process(result)
}

If you define a variable for "result" the following code will also be matched,

function test(otherToken) {
	message.post(otherToken)
	system.process(otherToken)
}

But this code would not be matched

function test(otherToken) {
	message.post(otherToken)
	system.process(result)
}

There are two types of variables. Those defined in self and those in scope.

  • self - enforces consistency of tokens in this node and its children
  • scope - If the node's parent also defines a variable with the same name then the value of this variable must match the one in its parent. If it's parent does not define a variable with the same name it behaves the same as self.

You can set variables like this:

.variables({
	variable1: 'self',
	variable2: 'scope'
})

Containers

Containers can be used to label blocks of code and set rules about what kind of code can go inside of them. A container is defined in the example snippet by adding an inline comment with the following format //:{container name}.

For each container in your snippet a rule for how to handle its children can be defined:

name matches when
any always
same-plus children from example snippet are present, in the order they appear in the example, plus any other children.
example snippet = [ A, B, C ]
[ B, C ] false
[ A, D, B, C ] true
[ A, B, C ] true
[ A, B, C, D ] true
same-plus-any-order children from the example snippet are present, in any order, plus any other children
example snippet = [ A, B, C ]
[ C, B, D, E, A ] true
[ A, B, C ] true
[ C, B, D, E ] false
same-any-order all children from the example snippet are present, in any order,
example snippet = [ A, B, C ]
[ B, C, A ] true
[ C, A, B ] true
[ C ] false
[ C, B, A, D ] false
exact children match those in the example snippet

Here's how you configure containers with the SDK:

js`
sendMessage('message', (response) => {
	//:response handler
}, (error) => {
	//:error handler
	throw new ResponseError(error)
})`

.containers({
	'response handler': 'any'
	'error handler': 'same-plus'
})

Once the containers were configured, this example would still be recodnized as valid:

sendMessage('message', (response) => {
	alert(response)
}, (error) => {
	retry(3, () => { // retry 3 times then throw
		throw new ResponseError(error)
	})
})

and this one would not:

sendMessage('message', (response) => {
	doSomething()
}, (error) => {
	return false
})

Customizing Abstraction

By default, Optic determines a basic abstraction schema for your generator based on the fields and finders you use when configuring the generator. However, sometimes you might want to a) customize that schema or b) have this generator implement a shared abstraction.

To customize the abstraction of your generator, just call .abstraction and pass a valid draft-4 JSON Schema. Here's an example schema that would require in to be one of the following values query, body, params, header.

.abstraction({
	"type": "object",
	"required": ["in", "name"],
	"properties": {
		"in": {
			"type": "string",
			"enum": ["query", "body", "params", "header"]
		},
		"name": {
			"type": "string"
		}
	}
})

If you want this generator to implement another abstraction just pass in the identifier. If the schema is coming from another package you need to add it to the dependencies section of your skill's declaration

.abstraction('optic:rest/parameter')

//skill declaration 
export default Skill('optic', 'testing', '0.4.0', {
	dependencies: {
		'optic:rest': '0.4.0'
	}
})

Testing

The Skills SDK makes it easy to write simple unit tests for your generators. To test a generator, first make sure it's registered withing the skill:

export default Skill('optic', 'testing', '0.4.0', {
	generators: [parameterGenerator]
})

Then get the test kit for the skill and the generator you want to test:

const skillTestKit = SkillTestKit(exampleSkill)
const parameterTestKit = skillTestKit.testGenerator('express-parameter')

You can test the functionality of the generator by calling: generate({input}), mutate('rawcode', {newInput}), and parse('rawcode')

describe('parameter generator', () => {
		const skillTestKit = SkillTestKit(exampleSkill)
		const parametersTestKit = expressSkillTestKit.testGenerator('express-parameter')
		
		it('can generate parameters', () => {
			const result = parametersTestKit.generate({in: 'query', name: 'testName'})
			assert(result.success)
			assert(result.code === 'req.query.testName')
		})

		it('can parse parameters', () => {
			const result = parametersTestKit.parse('req.body.param')
			assert(result.success)
			assert(result.value.in === 'body')
			assert(result.value.name === 'param')
		})

		it('will not parse parameters with invalid "in" values', () => {
			const result = parametersTestKit.parse('req.whazzzzup.param')
			assert(!result.success)
		})

		it('can mutate parameters', () => {
			const result = parametersTestKit.mutate('req.body.param', {in: 'query'})
			assert(result.code === 'req.query.param')
		})
})

Assuming your skill project was setup using opticsdk init, you can run npm run test to evaluate all the test suites in your project.

Next Steps