API Builder Tools 4.0.0 Release Note

API Builder 4.0.0 - 29 June 2018

API Builder V4.0 enables customers to install and run API Builder in a containerized environment on-premise. Previously, API Builder could only be installed in a cloud environment. For information on installing and getting started with API Builder V4.0, refer to the API Builder Getting Started Guide. The API Builder V4.0 technical preview release includes the following changes and new features to implement the ability to install API Builder on-premise. The release also includes fixed, known issues, and security vulnerabilities.

New features

  • RDPP-1243: Add flow input parameters and enable autocomplete.
  • RDPP-1251: Add a new asynchronous HTTP client flow-node plugin that supports calls to external services.
  • RDPP-4223: API Builder standalone can run and install on-premise with no Appcelerator cloud dependency.
  • RDPP-4225: Revise configuration files to be amenable to containers having no specific environment.
  • RDPP-4345: Load data connectors as plugins from node_modules.
  • RDPP-4348: Revise logging format to be amenable to container logging and write to stdout and not file.
  • RDPP-4369: Add a new Swagger flow-node plugin that supports orchestration to external services.
  • RDPP-4391: Migrate the MySQL data connector to plugin.
  • RDPP-4541: Migrate the MongoDB data connector to plugin.
  • RDPP-4543: Migrate the Oracle data connector to plugin.

Improvements

  • RDPP-1243: Previously, flows had no inputs and would receive implicit runtime parameters $.params, $.request, $.config, and $.env for use in runtime. Now, those parameters are explicitly part of the flow definition.

  • RDPP-1243: Previously, "Generate endpoints" from a model would generate flows that were implicit $.params, $.request, $.config, and $.env for use in runtime in the flow. Now, the flows only use the explicit API endpoint parameters needed to execute the model function, for example $.limit, $.where, and so forth.

  • RDPP-4045: Previously, the flow editor UI had a Save button that would save the current flow being edited and then exit to the list of API methods. Now, the flow editor UI has an Apply button that allows the flow to be saved, but stays in the flow editor and does not exit. The Cancel button has also been renamed to Close. If the flow has no changes, then Close will just exit. If the flow has changed, then Close will prompt to save or discard changes before exiting.
  • RDPP-4075: Previously, service connectors were installed into, and loaded from the ./serviceconnectors folder of your project. Now, they are installed as regular NPM modules into the ./node_modules folder using:

    npm install @axway/api-builder-plugin-sc-<name> 

    Where, <name> is the name of the service connector to install.

  • RDPP-4225: Previously, it was possible to specify environment variables from the operating system environment and apply them to the API Builder runtime (for example, ARROW_PORT). Now, it is no longer possible to automatically use ARROW_* environment properties. Instead, the configurable environment properties need to be explicitly included in the API Builder's configuration files (for example, conf/default.js), and reference the environment variable by name (for example, process.env.APP_PORT).

  • RDPP-4225: Previously, it was possible to have multiple sets of configuration files, for example, "development", "production", and so forth. Now, only two sets of configuration files are recognized: "default" and "local" (for example, default.js, foo.default.js, local.js, and foo.local.js), where the local config files are ignored by Git and NPM.

  • RDPP-4346: Previously, flow-node projects created with the Axway Flow SDK had to synchronously export their flow-node specifications which prevented the use of asynchronous APIs, such as network calls or complex parsers, when generating the flow-node specifications. Now, API Builder allows the flow-node projects to export a Promise which will resolve with the flow-node specification, allowing for asynchronous loading of the flow-node specifications. Now, this is the default behavior in all new flow-node projects.
  • RDPP-4348: Previously, API Builder would write aggregated transactional logs to ./logs. Now, API Builder logs everything to the console and as a result, the logging in conf/default.js is no longer used. Keeping the logging configuration will have no effect on the application.
  • RDPP-4390: Previously, API Builder loaded data connectors from the connectors folder of your project or the node_modules/connectors folder. Now, API Builder loads connectors as API Builder plugins which need to be installed into the node_modules folder and follow the naming convention .api-builder-plugin* For additional data connector information, refer to API Builder Connectors.
  • RDPP-4390: Previously, data connector configuration would name the connector as the key; for example, appc.mysql, and then list specific configuration properties for the connector, including an optional alias. Now, the data connector configuration uses the alias as a key; for example, userdb and the configuration properties must also include the connector name. For example:

                    userdb: {
                            connector: '@axway/api-builder-plugin-dc-mysql',
                            connectionPooling: true,
                            connectionLimit: 10,
                            host: 'localhost',
                            port: 3306,
                            database: 'userdb',
                            user: 'root',
                            password: 'root',
    
                            // Create models based on your schema that can be used in your API.
                            generateModelsFromSchema: true,
    
                            // Whether or not to generate APIs based on the methods in generated models.
                            modelAutogen: false
                    }
  • RDPP-4391: Previously, the appc.mysql data connector was installed into your project by the Appcelerator CLI. Now, the connector has been renamed and is published in NPM as @axway/api-builder-plugin-dc-mysql and NPM can be used to install the connector into your project.

  • RDPP-4392: Previously, the appc.composite connector was installed separately into your application. Now, the connector has been renamed to composite and is now integrated by default into your application and there is no need to install it separately.
  • RDPP-4524: Previously, when a flow was invoked it would log the step-by-step execution and include the flow-node id. The problem with this is that the flow-node IDs are not very informative or useful for identifying flow-nodes. For example, if you had two Compose flow-nodes (dot flow-nodes) they would log as dot.1 and dot.2. Now, logging includes the flow-node name. So if the user has named a Compose flow-node "My formatter", when the flow-node is invoked it will log as My formatter (dot.1).
  • RDPP-4541: Previously, the appc.mongo data connector was installed into your project by the appc CLI. Now, the connector has been renamed and is published in NPM as @axway/api-builder-plugin-dc-mongo and NPM can be used to install the connector into your project.
  • RDPP-4577: Previously, the Swagger generated for model CRUD APIs accepted additional properties in the request. This could lead to errors and unexpected behavior. Now, the APIs do not accept additional properties and they precisely define their return type in the id field.
  • RDPP-4639: Previously, flow-nodes were loaded from the /nodes directory or from the /node_modules directory with the prefix nodehandler-. Now, flow-nodes are regular plugins and should be prefixed api-builder-plugin-. Flow-nodes are no longer loaded from the /nodes directory. Flow-nodes generated with axway-flow-sdk 1.x will need to be upgraded to plugins.

  • RDPP-4639: Previously, data-connectors could be loaded from the /connectors directory. Now, since data-connectors are plugins which are regular node modules, they are only loaded from the node_modules directory.

  • RDPP-4639: Previously, four flow-nodes were available to API Builder projects - nodehandler-base64, nodehandler-dot, nodehandler-json, and nodehandler-restclient. These will continue to work with API Builder 3.0, Now, these same flow-nodes are available for API Builder 4.0.0 in the new plugin format under the following names: @axway/api-builder-plugin-fn-base64, @axway/api-builder-plugin-fn-dot, @axway/api-builder-plugin-fn-json, and @axway/api-builder-plugin-fn-restclient.

  • RDPP-4781: Previously, the API Builder runtime was in a module called arrow and was referred to as Arrow. Now, the API Builder runtime is in a module called @axway/api-builder-runtime and is referred to as APIBuilder.
  • RDPP-4781: Previously, the API Builder React engine was in a module named arrow-react-engine. Now, the API Builder React engine is in a module called @axway/api-builder-react-engine.

Fixed issues

  • RDPP-4128: Previously, service connectors with hyphenated names (for example, tenant-provisioning) would generate invalid JSON path selectors in the flow UI (for example, $.tenant-provisioning.provisionTenant.201). Now, service connectors with hyphenated names generate valid JSON path selectors.
  • RDPP-4137: Previously, the communication between the API Builder and the Service Connectors was implemented improperly in terms that service connectors did not return the expected structure. Now, each invocation of the function part of a service connector returns the expected structure having all - statusCode, headers, and data; for example, {"statusCode": 200, "headers": {}, "data": {"count": 4}}.
  • RDPP-4151: Previously, service-connectors could be set to invalid default output selectors when used in the flow editor, such as $.Marketo.getLists.400. Now, newly generated service connectors will set default output selectors to either $.response or $.error based on the status code of the response.
  • RDPP-4152: Previously, the methods on the model flow-nodes outputted Arrow.Model objects to the Flow context. Interacting with these in subsequent flow-nodes required a knowledge of the internal workings of the Model object. Now, the model flow-nodes output plain data objects containing the fields and values from the model method. This will only cause issues for flows that were expecting the Model object rather than a data object.

  • RDPP-4350: Previously, importing API endpoints into API Builder would incorrectly apply a Content-Type to GET methods. Now, imported endpoints do not apply Content-Type to GET methods.
  • RDPP-4350: Previously, API Builder would apply consumes, produces, tags, schemes, and security properties on the service's Swagger document. This would unintentionally mean that any endpoints without these properties defined per-path, would take these global values instead of their intended value. Now, any endpoints which are imported with any of these properties will only have them applied to that endpoint's methods, and API Builder will not apply any of these properties globally which could change the meaning of the resulting Swagger document.
  • RDPP-4355: Previously, HTTP method verbs in the API Documentation page for endpoints were lower-case. Now, they are correctly displayed in upper-case.
  • RDPP-4363: Previously, importing a Swagger document with an empty title would save the file to endpoints/*.json. This is because the file name is derived from the title. Also, if a file already existed with the desired name, an error would be thrown. Now, the imported file name or URL basename will be used if the Swagger title is blank. Additionally, if the desired filename already exists, then the imported filename will have an integer appended to it to ensure uniqueness.

  • RDPP-4368: Previously, calling query on the MySQL connector with sel containing an unknown field would have indeterminate results and would be potentially dangerous for SQL injection attacks. Now, unknown fields are filtered out and will return an error.
  • RDPP-4368: Previously, calling query on the MySQL connector with order containing unknown columns would have indeterminate results and was open to SQL injection attacks. Now, using order with an unknown column will return an error.
  • RDPP-4368: Previously, calling distinct on the MySQL connector with an unknown field would have indeterminate results and would be potentially dangerous for SQL injection attacks. Now, an unknown field will return an error.
  • RDPP-4368: Previously, calling query, distinct, count, or findAll on the MySQL connector with a where option that contained unknown columns would have indeterminate results and was open to SQL injection attacks. Now, using the where option with an unknown column will return an error.
  • RDPP-4368: Previously, calling distinct or findAll on the MySQL connector would not honor options for order, skip, or limit. Now, these options are honored.
  • RDPP-4368: Previously, calling upsert on the MySQL connector with an un-escaped ID value would have indeterminate results and would be potentially dangerous for SQL injection attacks. Now, ID is passed as a placeholder and escaped.
  • RDPP-4368: Previously, the model generated API for distinct had options for sel and unsel that were ignored. Now, these options are removed from the API and are ignored.
  • RDPP-4368: Previously, the distinct method on flow-nodes for connectors had parameters for sel and unsel that were ignored. Now, these parameters are removed from the model flow-node.
  • RDPP-4519: Previously, the CLI would suggest a regular NPM install when creating a new app. This may end up displaying node-gyp errors which can be ignored due to being related to optional dependencies. Most of the time, these can be skipped, and install time can be increased by not installing optional dependencies. Now, the CLI prompts the user to NPM install with the --no-optional flag. The Docker file bundled with new apps also uses --no-optional as part of the build.
  • RDPP-4531: Previously, query parameters in the Swagger generated from models were missing some maximum, minimum, and default properties. These are used to set parameter defaults and to enforce correct user input. For example, the page query parameter should be set to a minimum value of 1. Now, these values are part of the generated query parameters.

  • RDPP-4548: Previously, the connector did not support the MySQL decimal type. Now, the connector adds support for decimal table fields and converts them to a number in the corresponding JavaScript model.
  • RDPP-4550: Previously, endpoints with query parameters accepted page and per_page as well as skip and limit. The problem is that it wasn't clear how to effectively use these. To exacerbate the issue, supplying both sets of parameters would yield unexpected result ranges. Now, page and per_page parameters have been removed since their functionality is a subset of what can be provided by the skip and limit parameters. In the backend, page and per_page are computed via skip and limit so connectors that rely on them should continue to function.
  • RDPP-4559: Previously, when running an API Builder project that began with the character "u" on Microsoft Windows, the service would fail to start with a SyntaxError: Invalid Unicode escape sequence. Now, these projects will start correctly.
  • RDPP-4578: Previously, after importing a Swagger API, when trying to invoke an API method that consumes a body, where the consumes parameter was not defined in the original Swagger document (either on the method, or globally), API Builder would return "Request validation failed: Parameter (body) failed schema validation". Now, API Builder applies a default consumes parameter of application/json. The Swagger 2.0 specification does not explicitly handle this condition but their own tooling applies application/json and it is a reasonable assumption that the method can default to application/json when the body must validate against a JSON schema.
  • RDPP-4602: Previously, model fields with default values would get set in the model schema without validation or interpretation. Now, the default values are validated to ensure that they are cast as the correct type.
  • RDPP-4700: Previously, the distinct method for Mongo database collections did not honor skip, limit, or order query parameters. Now, the distinct method for Mongo database collections honors skip, limit, or order query parameters.
  • RDPP-4712: Previously, models would auto-generated API with optional body parameters for the Create, Upsert, Update, and Find and Modify methods. Now, the body parameters are required.
  • RDPP-4732: Previously, there was a serialization.exposePrimaryKeyAsId configuration option that was intended to force the primary key of a model to always be called id. Now, this configuration option has been removed and the model primary key field will always be exposed using its actual column name.
  • RDPP-4732: Previously, the primary keys on models had to be named id. Now, API Builder supports primary key columns with names other than id.
  • RDPP-4773: Previously, the distinct method for Mongo database collections did not honor skip, limit, or order query parameters. Now, the distinct method for Mongo database collections honors skip, limit, or order query parameters.

Known issues

  • RDPP-3825: Filtering the API Builder Console administrator access using IPv6 addresses may cause ENOTFOUND errors.
  • RDPP-3867: When attempting to create and save a flow for an imported Swagger endpoint that contains a path or paths defined by references the save will fail.
  • RDPP-3960: The API Builder Console does not recognize a required consumes value for form parameters if it is appended and the endpoint load will fail. For example:

    "consumes": [
    "multipart/form-data; charset=utf-8"
    ],

    The appended character set (charset=utf-8) will cause the endpoint load to fail.

  • RDPP-3979: When deleting endpoints which contain references within paths, a Page Not Found (404) error may be displayed in the API Builder Console. For example, a Swagger document with references within paths may look similar to this:

    {
    	"swagger": "2.0",
    	"paths" {
    		"x-path": {
    			"get": {}
    		},
    		"/find": {
    			"$ref": "#/paths/x-path"
    		},
    		"/search": {
    			"$ref": "#/paths/x-path"
    		}
    
    	}
    }

    The API Builder Console will fail to find GET /find since it is inside a $ref. If the API Builder Console has modified the referenced $ref, it could cause unexpected behavior for other paths referencing #/paths/x-path such as /search - deleting GET /find could unexpectedly delete GET /search too. 

  • RDPP-4050: When rendering the flow editor, the API Builder Console may fail to render the Scalable Vector Graphics (SVG) icons correctly in the Firefox browser. The render failure may result in blank icons being displayed in the tool panel and in the flow diagram. This is due to a long-standing bug in Firefox that fails to scale SVG graphics correctly. To fix, edit the SVG icon and add height and width. For example: <svg ... height="80" width="80" />
  • RDPP-4280: Editing large object parameters on the API Orchestration page in the API Builder Console may cause multiple, confusing flow-node configuration panel scrollbars to appear.
  • RDPP-4445: If a Swagger definition uses the allOf parameter, the body generated in the method test window is incorrect.
  • RDPP-4532: Given a data connector that generates models from a database, and is configured to auto-generate the API for those models (modelAutogen is set to true), and there exists a table with a primary key that is not an auto-incremented number and not named "id" (for example, is a PK on username), then the API generated will not properly handle the methods for Upsert or Update. In the UI, Upsert will render an example body with the non-id PK (for example, username) and id and will also fail 500 with various error messages depending on content of the body, "You must provide a Model id and data Object, that will be persisted", required body parameter: username missing", "You must provide a Model id and data Object, that will be persisted". Update will fail with 500 error "invalid field: id". Given a table "geozip", postalCode (PK string), lat (string), lng (string), the generated API methods disallows Create or Upsert.
  • RDPP-4716: Given a MySQL data connector that generates models from a database, and is configured to auto-generate the API for those models (modelAutogen is set to true), and there exists a table with no primary key, then the API Builder will not be able to handle the following methods: Update, Delete, Find By ID, Find and Modify, and Upsert.
  • RDPP-4724: Given an Oracle data connector that generates models from a database, and is configured to auto-generate the API for those models (modelAutogen is set to true), and there exists a table with no primary key, then the API Builder will not be able to handle the following methods: UpdateDeleteFind By IDFind and Modify, and Upsert.
  • RDPP-4727: When executing a count query on a composite model that is being sourced from an Oracle DB Model the reported result count will never exceed 10 irrespective of how many actual rows there are.
  • RDPP-4736: Given a Swagger with an extension, for example, on the paths object, the Swagger flow-node plugin can fail to load the Swagger file, resulting in an error: "Error loading plugin: @axway/api-builder-plugin-fn-swagger. Cannot convert undefined or null to object."
  • RDPP-4748: The Swagger document describes the application's configured scheme, host, and port; however, the application's scheme, host, and port are not editable via the application's configuration.
  • RDPP-4752: The format of a distinct query's response depends on the type of the connector.
  • RDPP-4759: Calling Update or Find and Modify on a Model that uses the composite connector and contains the required fields may fail and cause the server to terminate.
  • RDPP-4795: The Mongo database connector does not work well with public keys (PKs) that are not ObjectID.
  • RDPP-4813: If the endpoint Swagger file in the /endpoints directory contains special characters in its name, for example [test].json, the endpoint is not rendered correctly in the UI.

Security vulnerabilities

  • RDPP-4810: JSONSelect

    • Vulnerability: JSONSelect@0.4.0 is used indirectly by the api-builder-runtime. JSON path is used in the flow engine and flow editor to query object value(s) at runtime. The issue identified by the CVE is with a jQuery artifact that is vulnerable to XSS that is bundled with JSONSelect. jQuery version 1.6.1 has the following relevant CVEs reported against it:

    • Analysis: The JSONSelect npm package includes in its distribution, a copy of jQuery that is used for unit-test and is not used by JSONSelect itself, and as a result, it is not a risk for API Builder.
  • RDPP-4811: Bootstrap 20184

    • Vulnerability: API Builder UI uses react-bootstrap which has a dependency on Bootstrap v3.3.7, and it is 3.3.7 that contains the vulnerability. At the time of writing this ticket, Bootstrap did not publish a 3.x version after 3.3.7, so no fix was ever released. The Bootstrap library is now at 4.x. However, the react-bootstrap library that API Builder uses is actively working on a version compatible with Bootstrap v4.x. The following issue is logged against Bootstrap v3.3.7:
      • 20184 - XSS in data-target attribute
    • Analysis: The vulnerability and risk are documented in 20184. The API Builder UI only runs on the developer machine and is locked to localhost by default. The API Builder UI will not be installed in production. Furthermore, the UI bundled with API Builder does not use data-target attributes. The risk to API Builder is low.

Related Links